Group PJMED_VID_STRM
- group PJMED_VID_STRM
Video communication via the network.
A video stream is a bidirectional video communication between two endpoints. It corresponds to a video media description (“m=video” line) in SDP session descriptor.
A video stream consists of two unidirectional channels:
encoding channel, which transmits unidirectional video to remote, and
decoding channel, which receives unidirectional media from remote.
A video stream exports two media port interface (see Media Ports Framework), one for each direction, and application normally uses this interface to interconnect the stream to other PJMEDIA components, e.g: the video capture port supplies frames to the encoding port and video renderer consumes frames from the decoding port.
A video stream internally manages the following objects:
an instance of video codec (see Video Codecs),
two instances of RTP sessions (pjmedia_rtp_session, one for each direction),
one instance of RTCP session (pjmedia_rtcp_session),
and a reference to video transport to send and receive packets to/from the network (see Media Transport).
Video streams are created by calling pjmedia_vid_stream_create(), specifying pjmedia_stream_info structure in the parameter. Application can construct the pjmedia_vid_stream_info structure manually, or use pjmedia_vid_stream_info_from_sdp() function to construct the pjmedia_vid_stream_info from local and remote SDP session descriptors.
Typedefs
-
typedef struct pjmedia_vid_stream pjmedia_vid_stream
Enums
-
enum pjmedia_vid_stream_rc_method
Enumeration of video stream sending rate control.
Values:
-
enumerator PJMEDIA_VID_STREAM_RC_NONE
No sending rate control. All outgoing RTP packets will be transmitted immediately right after encoding process is done.
-
enumerator PJMEDIA_VID_STREAM_RC_SIMPLE_BLOCKING
Simple blocking. Each outgoing RTP packet transmission may be delayed to avoid peak bandwidth that is much higher than specified. The thread invoking the video stream put_frame(), e.g: video capture device thread, will be blocked whenever transmission delay takes place.
-
enumerator PJMEDIA_VID_STREAM_RC_NONE
Functions
-
pj_status_t pjmedia_vid_stream_info_from_sdp(pjmedia_vid_stream_info *si, pj_pool_t *pool, pjmedia_endpt *endpt, const pjmedia_sdp_session *local, const pjmedia_sdp_session *remote, unsigned stream_idx)
This function will initialize the video stream info based on information in both SDP session descriptors for the specified stream index. The remaining information will be taken from default codec parameters. If socket info array is specified, the socket will be copied to the session info as well.
- Parameters:
si – Stream info structure to be initialized.
pool – Pool to allocate memory.
endpt – PJMEDIA endpoint instance.
local – Local SDP session descriptor.
remote – Remote SDP session descriptor.
stream_idx – Media stream index in the session descriptor.
- Returns:
PJ_SUCCESS if stream info is successfully initialized.
-
void pjmedia_vid_stream_rc_config_default(pjmedia_vid_stream_rc_config *cfg)
Initialize the video stream rate control with default settings.
- Parameters:
cfg – Video stream rate control structure to be initialized.
-
void pjmedia_vid_stream_sk_config_default(pjmedia_vid_stream_sk_config *cfg)
Initialize the video stream send keyframe with default settings.
- Parameters:
cfg – Video stream send keyframe structure to be initialized.
-
pj_status_t pjmedia_vid_stream_create(pjmedia_endpt *endpt, pj_pool_t *pool, pjmedia_vid_stream_info *info, pjmedia_transport *tp, void *user_data, pjmedia_vid_stream **p_stream)
Create a video stream based on the specified parameter. After the video stream has been created, application normally would want to get the media port interface of the stream, by calling pjmedia_vid_stream_get_port(). The media port interface exports put_frame() and get_frame() function, used to transmit and receive media frames from the stream.
Without application calling put_frame() and get_frame(), there will be no media frames transmitted or received by the stream.
- Parameters:
endpt – Media endpoint.
pool – Optional pool to allocate memory for the stream. If this is not specified, one will be created internally. A large number of memory may be needed because jitter buffer needs to preallocate some storage.
info – Stream information to create the stream. Upon return, this info will be updated with the information from the instantiated codec. Note that if the “pool” argument is NULL, some fields in this “info” parameter will be allocated from the internal pool of the stream, which means that they will only remain valid as long as the stream is not destroyed.
tp – Media transport instance used to transmit and receive RTP/RTCP packets to/from the underlying network.
user_data – Arbitrary user data (for future callback feature).
p_stream – Pointer to receive the video stream.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_destroy(pjmedia_vid_stream *stream)
Destroy the video stream.
- Parameters:
stream – The video stream.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_get_port(pjmedia_vid_stream *stream, pjmedia_dir dir, pjmedia_port **p_port)
Get the media port interface of the stream. The media port interface declares put_frame() and get_frame() function, which is the only way for application to transmit and receive media frames from the stream. As bidirectional video streaming may have different video formats in the encoding and decoding direction, there are two media ports exported by the video stream, one for each direction.
- Parameters:
stream – The video stream.
dir – The video direction.
p_port – Pointer to receive the port interface.
- Returns:
PJ_SUCCESS on success.
-
pjmedia_transport *pjmedia_vid_stream_get_transport(pjmedia_vid_stream *st)
Get the media transport object associated with this stream.
- Parameters:
st – The video stream.
- Returns:
The transport object being used by the stream.
-
pj_status_t pjmedia_vid_stream_get_stat(const pjmedia_vid_stream *stream, pjmedia_rtcp_stat *stat)
Get the stream statistics. See also pjmedia_stream_get_stat_jbuf()
- Parameters:
stream – The video stream.
stat – Media stream statistics.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_reset_stat(pjmedia_vid_stream *stream)
Reset the video stream statistics.
- Parameters:
stream – The video stream.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_get_stat_jbuf(const pjmedia_vid_stream *stream, pjmedia_jb_state *state)
Get current jitter buffer state. See also pjmedia_stream_get_stat()
- Parameters:
stream – The video stream.
state – Jitter buffer state.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_get_info(const pjmedia_vid_stream *stream, pjmedia_vid_stream_info *info)
Get the stream info.
- Parameters:
stream – The video stream.
info – Video stream info.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_start(pjmedia_vid_stream *stream)
Start the video stream. This will start the appropriate channels in the video stream, depending on the video direction that was set when the stream was created.
- Parameters:
stream – The video stream.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_modify_codec_param(pjmedia_vid_stream *stream, const pjmedia_vid_codec_param *param)
Modify the video stream’s codec parameter after the codec is opened. Note that not all codec backends support modifying parameters during runtime and only certain parameters can be changed.
Currently, only Video Toolbox and OpenH264 backends support runtime adjustment of encoding bitrate (avg_bps and max_bps).
- Parameters:
stream – The video stream.
param – The new codec parameter.
- Returns:
PJ_SUCCESS on success.
-
pj_bool_t pjmedia_vid_stream_is_running(pjmedia_vid_stream *stream, pjmedia_dir dir)
Query if the stream is started on the specified direction.
- Parameters:
stream – The video stream.
dir – The direction to be checked.
- Returns:
PJ_TRUE if stream is started.
-
pj_status_t pjmedia_vid_stream_pause(pjmedia_vid_stream *stream, pjmedia_dir dir)
Pause stream channels.
- Parameters:
stream – The video stream.
dir – Which channel direction to pause.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_resume(pjmedia_vid_stream *stream, pjmedia_dir dir)
Resume stream channels.
- Parameters:
stream – The video stream.
dir – Which channel direction to resume.
- Returns:
PJ_SUCCESS on success;
-
pj_status_t pjmedia_vid_stream_send_keyframe(pjmedia_vid_stream *stream)
Force stream to send video keyframe on the next transmission.
- Parameters:
stream – The video stream.
- Returns:
PJ_SUCCESS on success;
-
pj_status_t pjmedia_vid_stream_send_rtcp_sdes(pjmedia_vid_stream *stream)
Send RTCP SDES for the video stream.
- Parameters:
stream – The video stream.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_send_rtcp_bye(pjmedia_vid_stream *stream)
Send RTCP BYE for the video stream.
- Parameters:
stream – The video stream.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_send_rtcp_pli(pjmedia_vid_stream *stream)
Send RTCP PLI for the video stream.
- Parameters:
stream – The video stream.
- Returns:
PJ_SUCCESS on success.
-
pj_status_t pjmedia_vid_stream_get_rtp_session_info(pjmedia_vid_stream *stream, pjmedia_stream_rtp_sess_info *session_info)
Get the RTP session information of the video media stream. This function can be useful for app with custom media transport to inject/filter some outgoing/incoming proprietary packets into normal video RTP traffics. This will return the original pointer to the internal states of the stream, and generally it is not advisable for app to modify them.
- Parameters:
stream – The video media stream.
session_info – The stream session info.
- Returns:
PJ_SUCCESS on success.
-
struct pjmedia_vid_stream_rc_config
- #include <vid_stream.h>
Structure of configuration settings for video stream sending rate control.
Public Members
-
pjmedia_vid_stream_rc_method method
Rate control method.
Default: PJMEDIA_VID_STREAM_RC_SIMPLE_BLOCKING.
-
unsigned bandwidth
Upstream/outgoing bandwidth. If this is set to zero, the video stream will use codec maximum bitrate setting.
Default: 0 (follow codec maximum bitrate).
-
pjmedia_vid_stream_rc_method method
-
struct pjmedia_vid_stream_sk_config
- #include <vid_stream.h>
Structure of configuration settings for video stream sending keyframe after it is created.
-
struct pjmedia_vid_stream_info
- #include <vid_stream.h>
This structure describes video stream information. Each video stream corresponds to one “m=” line in SDP session descriptor, and it has its own RTP/RTCP socket pair.
Public Members
-
pjmedia_type type
Media type (audio, video)
-
pjmedia_tp_proto proto
Transport protocol (RTP/AVP, etc.)
-
pjmedia_dir dir
Media direction.
-
pj_sockaddr local_addr
Local RTP address
-
pj_sockaddr rem_addr
Remote RTP address
-
pj_sockaddr rem_rtcp
Optional remote RTCP address. If sin_family is zero, the RTP address will be calculated from RTP.
-
pjmedia_rtcp_fb_info loc_rtcp_fb
Local RTCP-FB info.
-
pjmedia_rtcp_fb_info rem_rtcp_fb
Remote RTCP-FB info.
-
unsigned tx_pt
Outgoing codec paylaod type.
-
unsigned rx_pt
Incoming codec paylaod type.
-
pj_uint32_t ssrc
RTP SSRC.
-
pj_uint32_t rem_ssrc
Remote RTP SSRC.
-
pj_uint32_t rtp_ts
Initial RTP timestamp.
-
pj_uint16_t rtp_seq
Initial RTP sequence number.
-
pj_uint8_t rtp_seq_ts_set
Bitmask flags if initial RTP sequence and/or timestamp for sender are set. bit 0/LSB : sequence flag bit 1 : timestamp flag
-
int jb_init
Jitter buffer init delay in msec.
(-1 for default).
-
int jb_min_pre
Jitter buffer minimum prefetch delay in msec (-1 for default).
-
int jb_max_pre
Jitter buffer maximum prefetch delay in msec (-1 for default).
-
int jb_max
Jitter buffer max delay in msec.
-
pjmedia_stream_ka_config ka_cfg
Stream send kep-alive settings.
-
pjmedia_vid_codec_info codec_info
Incoming codec format info.
-
pjmedia_vid_codec_param *codec_param
Optional codec param.
-
pjmedia_vid_stream_rc_config rc_cfg
Stream send rate control settings.
-
pjmedia_vid_stream_sk_config sk_cfg
Stream send keyframe settings.
-
pjmedia_type type