Group PJMED_VID_STRM

group PJMED_VID_STRM

Video communication via the network.

A video stream is a bidirectional video communication between two endpoints. It corresponds to a video media description (“m=video” line) in SDP session descriptor.

A video stream consists of two unidirectional channels:

  • encoding channel, which transmits unidirectional video to remote, and

  • decoding channel, which receives unidirectional media from remote.

A video stream exports two media port interface (see Media Ports Framework), one for each direction, and application normally uses this interface to interconnect the stream to other PJMEDIA components, e.g: the video capture port supplies frames to the encoding port and video renderer consumes frames from the decoding port.

A video stream internally manages the following objects:

Video streams are created by calling pjmedia_vid_stream_create(), specifying pjmedia_stream_info structure in the parameter. Application can construct the pjmedia_vid_stream_info structure manually, or use pjmedia_vid_stream_info_from_sdp() function to construct the pjmedia_vid_stream_info from local and remote SDP session descriptors.

Typedefs

typedef struct pjmedia_vid_stream pjmedia_vid_stream

Enums

enum pjmedia_vid_stream_rc_method

Enumeration of video stream sending rate control.

Values:

enumerator PJMEDIA_VID_STREAM_RC_NONE

No sending rate control. All outgoing RTP packets will be transmitted immediately right after encoding process is done.

enumerator PJMEDIA_VID_STREAM_RC_SIMPLE_BLOCKING

Simple blocking. Each outgoing RTP packet transmission may be delayed to avoid peak bandwidth that is much higher than specified. The thread invoking the video stream put_frame(), e.g: video capture device thread, will be blocked whenever transmission delay takes place.

Functions

pj_status_t pjmedia_vid_stream_info_from_sdp(pjmedia_vid_stream_info *si, pj_pool_t *pool, pjmedia_endpt *endpt, const pjmedia_sdp_session *local, const pjmedia_sdp_session *remote, unsigned stream_idx)

This function will initialize the video stream info based on information in both SDP session descriptors for the specified stream index. The remaining information will be taken from default codec parameters. If socket info array is specified, the socket will be copied to the session info as well.

Parameters:
  • si – Stream info structure to be initialized.

  • pool – Pool to allocate memory.

  • endpt – PJMEDIA endpoint instance.

  • local – Local SDP session descriptor.

  • remote – Remote SDP session descriptor.

  • stream_idx – Media stream index in the session descriptor.

Returns:

PJ_SUCCESS if stream info is successfully initialized.

void pjmedia_vid_stream_rc_config_default(pjmedia_vid_stream_rc_config *cfg)

Initialize the video stream rate control with default settings.

Parameters:

cfg – Video stream rate control structure to be initialized.

void pjmedia_vid_stream_sk_config_default(pjmedia_vid_stream_sk_config *cfg)

Initialize the video stream send keyframe with default settings.

Parameters:

cfg – Video stream send keyframe structure to be initialized.

pj_status_t pjmedia_vid_stream_create(pjmedia_endpt *endpt, pj_pool_t *pool, pjmedia_vid_stream_info *info, pjmedia_transport *tp, void *user_data, pjmedia_vid_stream **p_stream)

Create a video stream based on the specified parameter. After the video stream has been created, application normally would want to get the media port interface of the stream, by calling pjmedia_vid_stream_get_port(). The media port interface exports put_frame() and get_frame() function, used to transmit and receive media frames from the stream.

Without application calling put_frame() and get_frame(), there will be no media frames transmitted or received by the stream.

Parameters:
  • endpt – Media endpoint.

  • pool – Optional pool to allocate memory for the stream. If this is not specified, one will be created internally. A large number of memory may be needed because jitter buffer needs to preallocate some storage.

  • info – Stream information to create the stream. Upon return, this info will be updated with the information from the instantiated codec. Note that if the “pool” argument is NULL, some fields in this “info” parameter will be allocated from the internal pool of the stream, which means that they will only remain valid as long as the stream is not destroyed.

  • tp – Media transport instance used to transmit and receive RTP/RTCP packets to/from the underlying network.

  • user_data – Arbitrary user data (for future callback feature).

  • p_stream – Pointer to receive the video stream.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_destroy(pjmedia_vid_stream *stream)

Destroy the video stream.

Parameters:

stream – The video stream.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_get_port(pjmedia_vid_stream *stream, pjmedia_dir dir, pjmedia_port **p_port)

Get the media port interface of the stream. The media port interface declares put_frame() and get_frame() function, which is the only way for application to transmit and receive media frames from the stream. As bidirectional video streaming may have different video formats in the encoding and decoding direction, there are two media ports exported by the video stream, one for each direction.

Parameters:
  • stream – The video stream.

  • dir – The video direction.

  • p_port – Pointer to receive the port interface.

Returns:

PJ_SUCCESS on success.

pjmedia_transport *pjmedia_vid_stream_get_transport(pjmedia_vid_stream *st)

Get the media transport object associated with this stream.

Parameters:

st – The video stream.

Returns:

The transport object being used by the stream.

pj_status_t pjmedia_vid_stream_get_stat(const pjmedia_vid_stream *stream, pjmedia_rtcp_stat *stat)

Get the stream statistics. See also pjmedia_stream_get_stat_jbuf()

Parameters:
  • stream – The video stream.

  • stat – Media stream statistics.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_reset_stat(pjmedia_vid_stream *stream)

Reset the video stream statistics.

Parameters:

stream – The video stream.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_get_stat_jbuf(const pjmedia_vid_stream *stream, pjmedia_jb_state *state)

Get current jitter buffer state. See also pjmedia_stream_get_stat()

Parameters:
  • stream – The video stream.

  • state – Jitter buffer state.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_get_info(const pjmedia_vid_stream *stream, pjmedia_vid_stream_info *info)

Get the stream info.

Parameters:
  • stream – The video stream.

  • info – Video stream info.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_start(pjmedia_vid_stream *stream)

Start the video stream. This will start the appropriate channels in the video stream, depending on the video direction that was set when the stream was created.

Parameters:

stream – The video stream.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_modify_codec_param(pjmedia_vid_stream *stream, const pjmedia_vid_codec_param *param)

Modify the video stream’s codec parameter after the codec is opened. Note that not all codec backends support modifying parameters during runtime and only certain parameters can be changed.

Currently, only Video Toolbox and OpenH264 backends support runtime adjustment of encoding bitrate (avg_bps and max_bps).

Parameters:
  • stream – The video stream.

  • param – The new codec parameter.

Returns:

PJ_SUCCESS on success.

pj_bool_t pjmedia_vid_stream_is_running(pjmedia_vid_stream *stream, pjmedia_dir dir)

Query if the stream is started on the specified direction.

Parameters:
  • stream – The video stream.

  • dir – The direction to be checked.

Returns:

PJ_TRUE if stream is started.

pj_status_t pjmedia_vid_stream_pause(pjmedia_vid_stream *stream, pjmedia_dir dir)

Pause stream channels.

Parameters:
  • stream – The video stream.

  • dir – Which channel direction to pause.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_resume(pjmedia_vid_stream *stream, pjmedia_dir dir)

Resume stream channels.

Parameters:
  • stream – The video stream.

  • dir – Which channel direction to resume.

Returns:

PJ_SUCCESS on success;

pj_status_t pjmedia_vid_stream_send_keyframe(pjmedia_vid_stream *stream)

Force stream to send video keyframe on the next transmission.

Parameters:

stream – The video stream.

Returns:

PJ_SUCCESS on success;

pj_status_t pjmedia_vid_stream_send_rtcp_sdes(pjmedia_vid_stream *stream)

Send RTCP SDES for the video stream.

Parameters:

stream – The video stream.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_send_rtcp_bye(pjmedia_vid_stream *stream)

Send RTCP BYE for the video stream.

Parameters:

stream – The video stream.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_send_rtcp_pli(pjmedia_vid_stream *stream)

Send RTCP PLI for the video stream.

Parameters:

stream – The video stream.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_vid_stream_get_rtp_session_info(pjmedia_vid_stream *stream, pjmedia_stream_rtp_sess_info *session_info)

Get the RTP session information of the video media stream. This function can be useful for app with custom media transport to inject/filter some outgoing/incoming proprietary packets into normal video RTP traffics. This will return the original pointer to the internal states of the stream, and generally it is not advisable for app to modify them.

Parameters:
  • stream – The video media stream.

  • session_info – The stream session info.

Returns:

PJ_SUCCESS on success.

struct pjmedia_vid_stream_rc_config
#include <vid_stream.h>

Structure of configuration settings for video stream sending rate control.

Public Members

pjmedia_vid_stream_rc_method method

Rate control method.

Default: PJMEDIA_VID_STREAM_RC_SIMPLE_BLOCKING.

unsigned bandwidth

Upstream/outgoing bandwidth. If this is set to zero, the video stream will use codec maximum bitrate setting.

Default: 0 (follow codec maximum bitrate).

struct pjmedia_vid_stream_sk_config
#include <vid_stream.h>

Structure of configuration settings for video stream sending keyframe after it is created.

Public Members

unsigned count

The number of keyframe to be sent after the stream is created.

Default: PJMEDIA_VID_STREAM_START_KEYFRAME_CNT

unsigned interval

The keyframe sending interval after the stream is created.

Default: PJMEDIA_VID_STREAM_START_KEYFRAME_INTERVAL_MSEC

struct pjmedia_vid_stream_info
#include <vid_stream.h>

This structure describes video stream information. Each video stream corresponds to one “m=” line in SDP session descriptor, and it has its own RTP/RTCP socket pair.

Public Members

pjmedia_type type

Media type (audio, video)

pjmedia_tp_proto proto

Transport protocol (RTP/AVP, etc.)

pjmedia_dir dir

Media direction.

pj_sockaddr local_addr

Local RTP address

pj_sockaddr rem_addr

Remote RTP address

pj_sockaddr rem_rtcp

Optional remote RTCP address. If sin_family is zero, the RTP address will be calculated from RTP.

pj_bool_t rtcp_mux

Use RTP and RTCP multiplexing.

pjmedia_rtcp_fb_info loc_rtcp_fb

Local RTCP-FB info.

pjmedia_rtcp_fb_info rem_rtcp_fb

Remote RTCP-FB info.

unsigned tx_pt

Outgoing codec paylaod type.

unsigned rx_pt

Incoming codec paylaod type.

pj_uint32_t ssrc

RTP SSRC.

pj_str_t cname

RTCP CNAME.

pj_bool_t has_rem_ssrc

Has remote RTP SSRC?

pj_uint32_t rem_ssrc

Remote RTP SSRC.

pj_str_t rem_cname

Remote RTCP CNAME.

pj_uint32_t rtp_ts

Initial RTP timestamp.

pj_uint16_t rtp_seq

Initial RTP sequence number.

pj_uint8_t rtp_seq_ts_set

Bitmask flags if initial RTP sequence and/or timestamp for sender are set. bit 0/LSB : sequence flag bit 1 : timestamp flag

int jb_init

Jitter buffer init delay in msec.

(-1 for default).

int jb_min_pre

Jitter buffer minimum prefetch delay in msec (-1 for default).

int jb_max_pre

Jitter buffer maximum prefetch delay in msec (-1 for default).

int jb_max

Jitter buffer max delay in msec.

pj_bool_t use_ka

Stream keep-alive and NAT hole punch (see #PJMEDIA_STREAM_ENABLE_KA) is enabled?

pjmedia_stream_ka_config ka_cfg

Stream send kep-alive settings.

pjmedia_vid_codec_info codec_info

Incoming codec format info.

pjmedia_vid_codec_param *codec_param

Optional codec param.

pj_bool_t rtcp_sdes_bye_disabled

Disable automatic sending of RTCP SDES and BYE.

pjmedia_vid_stream_rc_config rc_cfg

Stream send rate control settings.

pjmedia_vid_stream_sk_config sk_cfg

Stream send keyframe settings.