Group PJSUA_LIB_MEDIA

group PJSUA_LIB_MEDIA

Media manipulation.

PJSUA has rather powerful media features, which are built around the PJMEDIA conference bridge. Basically, all media “ports” (such as calls, WAV players, WAV playlist, file recorders, sound device, tone generators, etc) are terminated in the conference bridge, and application can manipulate the interconnection between these terminations freely.

The conference bridge provides powerful switching and mixing functionality for application. With the conference bridge, each conference slot (e.g. a call) can transmit to multiple destinations, and one destination can receive from multiple sources. If more than one media terminations are terminated in the same slot, the conference bridge will mix the signal automatically.

Application connects one media termination/slot to another by calling pjsua_conf_connect() function. This will establish unidirectional media flow from the source termination to the sink termination. To establish bidirectional media flow, application wound need to make another call to pjsua_conf_connect(), this time inverting the source and destination slots in the parameter.

For example, to stream a WAV file to remote call, application may use the following steps:

pj_status_t stream_to_call( pjsua_call_id call_id )
{
   pjsua_player_id player_id;
   
   status = pjsua_player_create("mysong.wav", 0, &player_id);
   if (status != PJ_SUCCESS)
      return status;

   status = pjsua_conf_connect( pjsua_player_get_conf_port(),
                         pjsua_call_get_conf_port() );
}

Other features of PJSUA media:

  • efficient N to M interconnections between media terminations.

  • media termination can be connected to itself to create loopback media.

  • the media termination may have different clock rates, and resampling will be done automatically by conference bridge.

  • media terminations may also have different frame time; the conference bridge will perform the necessary bufferring to adjust the difference between terminations.

  • interconnections are removed automatically when media termination is removed from the bridge.

  • sound device may be changed even when there are active media interconnections.

  • correctly report call’s media quality (in pjsua_call_dump()) from RTCP packet exchange.

Defines

PJSUA_MEDIA_HAS_PJMEDIA

Use PJMEDIA for media? Set this to zero when using third party media stack.

PJSUA_THIRD_PARTY_STREAM_HAS_GET_INFO

Specify whether the third party stream has the capability of retrieving the stream info, i.e: pjmedia_stream_get_info() and pjmedia_vid_stream_get_info(). Currently this capability is required by smart media update and call dump.

PJSUA_THIRD_PARTY_STREAM_HAS_GET_STAT

Specify whether the third party stream has the capability of retrieving the stream statistics, i.e: pjmedia_stream_get_stat() and pjmedia_vid_stream_get_stat(). Currently this capability is required by call dump.

PJSUA_MAX_CONF_PORTS

Max ports in the conference bridge. This setting is the default value for pjsua_media_config.max_media_ports.

PJSUA_DEFAULT_CLOCK_RATE

The default clock rate to be used by the conference bridge. This setting is the default value for pjsua_media_config.clock_rate.

PJSUA_DEFAULT_AUDIO_FRAME_PTIME

Default frame length in the conference bridge. This setting is the default value for pjsua_media_config.audio_frame_ptime.

PJSUA_DEFAULT_CODEC_QUALITY

Default codec quality settings. This setting is the default value for pjsua_media_config.quality.

PJSUA_DEFAULT_ILBC_MODE

Default iLBC mode. This setting is the default value for pjsua_media_config.ilbc_mode.

PJSUA_DEFAULT_EC_TAIL_LEN

The default echo canceller tail length. This setting is the default value for pjsua_media_config.ec_tail_len.

PJSUA_MAX_PLAYERS

The maximum file player.

PJSUA_MAX_RECORDERS

The maximum file player.

PJSUA_SDP_SESS_HAS_CONN

Enable/disable “c=” line in SDP session level. Set to zero to disable it.

PJSUA_TRANSPORT_RESTART_DELAY_TIME

Specify the delay needed when restarting the transport/listener. e.g: 10 msec on Linux or Android, and 0 on the other platforms.

Typedefs

typedef struct pjsua_ext_snd_dev pjsua_ext_snd_dev

Opaque type of extra sound device, an additional sound device beside the primary sound device (the one instantiated via pjsua_set_snd_dev() or pjsua_set_snd_dev2()). This sound device is also registered to conference bridge so it can be used as a normal conference bridge port, e.g: connect it to/from other ports, adjust/check audio level, etc. The conference bridge port ID can be queried using pjsua_ext_snd_dev_get_conf_port().

Application may also use this API to improve media clock. Normally media clock is driven by sound device in master port, but unfortunately some sound devices may produce jittery clock. To improve media clock, application can install Null Sound Device (i.e: using pjsua_set_null_snd_dev()), which will act as a master port, and instantiate the sound device as extra sound device.

Note that extra sound device will not have auto-close upon idle feature. Also note that currently extra sound device only supports mono channel.

Enums

enum pjsua_snd_dev_id

Sound device index constants.

Values:

PJSUA_SND_DEFAULT_CAPTURE_DEV = PJMEDIA_AUD_DEFAULT_CAPTURE_DEV

Constant to denote default capture device.

PJSUA_SND_DEFAULT_PLAYBACK_DEV = PJMEDIA_AUD_DEFAULT_PLAYBACK_DEV

Constant to denote default playback device.

PJSUA_SND_NO_DEV = PJMEDIA_AUD_INVALID_DEV

Constant to denote that no sound device is being used.

PJSUA_SND_NULL_DEV = -99

Constant to denote null sound device.

enum pjsua_snd_dev_mode

This enumeration specifies the sound device mode.

Values:

PJSUA_SND_DEV_SPEAKER_ONLY = 1

Open sound device without mic (speaker only).

PJSUA_SND_DEV_NO_IMMEDIATE_OPEN = 2

Do not open sound device, after setting the sound device.

Functions

void pjsua_media_config_default(pjsua_media_config *cfg)

Use this function to initialize media config.

Parameters
  • cfg: The media config to be initialized.

void pjsua_snd_dev_param_default(pjsua_snd_dev_param *prm)

Initialize pjsua_snd_dev_param with default values.

Parameters
  • prm: The parameter.

void pjsua_conf_connect_param_default(pjsua_conf_connect_param *prm)

Initialize pjsua_conf_connect_param with default values.

Parameters
  • prm: The parameter.

unsigned pjsua_conf_get_max_ports(void)

Get maxinum number of conference ports.

Return

Maximum number of ports in the conference bridge.

unsigned pjsua_conf_get_active_ports(void)

Get current number of active ports in the bridge.

Return

The number.

pj_status_t pjsua_enum_conf_ports(pjsua_conf_port_id id[], unsigned *count)

Enumerate all conference ports.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • id: Array of conference port ID to be initialized.

  • count: On input, specifies max elements in the array. On return, it contains actual number of elements that have been initialized.

pj_status_t pjsua_conf_get_port_info(pjsua_conf_port_id port_id, pjsua_conf_port_info *info)

Get information about the specified conference port

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • port_id: Port identification.

  • info: Pointer to store the port info.

pj_status_t pjsua_conf_add_port(pj_pool_t *pool, pjmedia_port *port, pjsua_conf_port_id *p_id)

Add arbitrary media port to PJSUA’s conference bridge. Application can use this function to add the media port that it creates. For media ports that are created by PJSUA-LIB (such as calls, file player, or file recorder), PJSUA-LIB will automatically add the port to the bridge.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • pool: Pool to use.

  • port: Media port to be added to the bridge.

  • p_id: Optional pointer to receive the conference slot id.

pj_status_t pjsua_conf_remove_port(pjsua_conf_port_id port_id)

Remove arbitrary slot from the conference bridge. Application should only call this function if it registered the port manually with previous call to pjsua_conf_add_port().

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • port_id: The slot id of the port to be removed.

pj_status_t pjsua_conf_connect(pjsua_conf_port_id source, pjsua_conf_port_id sink)

Establish unidirectional media flow from souce to sink. One source may transmit to multiple destinations/sink. And if multiple sources are transmitting to the same sink, the media will be mixed together. Source and sink may refer to the same ID, effectively looping the media.

If bidirectional media flow is desired, application needs to call this function twice, with the second one having the arguments reversed.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • source: Port ID of the source media/transmitter.

  • sink: Port ID of the destination media/received.

pj_status_t pjsua_conf_connect2(pjsua_conf_port_id source, pjsua_conf_port_id sink, const pjsua_conf_connect_param *prm)

Establish unidirectional media flow from source to sink. One source may transmit to multiple destinations/sink. And if multiple sources are transmitting to the same sink, the media will be mixed together. Source and sink may refer to the same ID, effectively looping the media.

Signal level from the source to the sink can be adjusted by making it louder or quieter via the parameter param. The level adjustment will apply to a specific connection only (i.e. only for the signal from the source to the sink), as compared to pjsua_conf_adjust_tx_level()/pjsua_conf_adjust_rx_level() which applies to all signals from/to that port. The signal adjustment will be cumulative, in this following order: signal from the source will be adjusted with the level specified in pjsua_conf_adjust_rx_level(), then with the level specified via this API, and finally with the level specified to the sink’s pjsua_conf_adjust_tx_level().

If bidirectional media flow is desired, application needs to call this function twice, with the second one having the arguments reversed.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • source: Port ID of the source media/transmitter.

  • sink: Port ID of the destination media/received.

  • prm: Conference port connection param. If set to NULL, default values will be used.

pj_status_t pjsua_conf_disconnect(pjsua_conf_port_id source, pjsua_conf_port_id sink)

Disconnect media flow from the source to destination port.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • source: Port ID of the source media/transmitter.

  • sink: Port ID of the destination media/received.

pj_status_t pjsua_conf_adjust_tx_level(pjsua_conf_port_id slot, float level)

Adjust the signal level to be transmitted from the bridge to the specified port by making it louder or quieter.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • slot: The conference bridge slot number.

  • level: Signal level adjustment. Value 1.0 means no level adjustment, while value 0 means to mute the port.

pj_status_t pjsua_conf_adjust_rx_level(pjsua_conf_port_id slot, float level)

Adjust the signal level to be received from the specified port (to the bridge) by making it louder or quieter.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • slot: The conference bridge slot number.

  • level: Signal level adjustment. Value 1.0 means no level adjustment, while value 0 means to mute the port.

pj_status_t pjsua_conf_get_signal_level(pjsua_conf_port_id slot, unsigned *tx_level, unsigned *rx_level)

Get last signal level transmitted to or received from the specified port. The signal level is an integer value in zero to 255, with zero indicates no signal, and 255 indicates the loudest signal level.

Return

PJ_SUCCESS on success.

Parameters
  • slot: The conference bridge slot number.

  • tx_level: Optional argument to receive the level of signal transmitted to the specified port (i.e. the direction is from the bridge to the port).

  • rx_level: Optional argument to receive the level of signal received from the port (i.e. the direction is from the port to the bridge).

pj_status_t pjsua_player_create(const pj_str_t *filename, unsigned options, pjsua_player_id *p_id)

Create a file player, and automatically add this player to the conference bridge.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • filename: The filename to be played. Currently only WAV files are supported, and the WAV file MUST be formatted as 16bit PCM mono/single channel (any clock rate is supported).

  • options: Optional option flag. Application may specify PJMEDIA_FILE_NO_LOOP to prevent playback loop.

  • p_id: Pointer to receive player ID.

pj_status_t pjsua_playlist_create(const pj_str_t file_names[], unsigned file_count, const pj_str_t *label, unsigned options, pjsua_player_id *p_id)

Create a file playlist media port, and automatically add the port to the conference bridge.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • file_names: Array of file names to be added to the play list. Note that the files must have the same clock rate, number of channels, and number of bits per sample.

  • file_count: Number of files in the array.

  • label: Optional label to be set for the media port.

  • options: Optional option flag. Application may specify PJMEDIA_FILE_NO_LOOP to prevent looping.

  • p_id: Optional pointer to receive player ID.

pjsua_conf_port_id pjsua_player_get_conf_port(pjsua_player_id id)

Get conference port ID associated with player or playlist.

Return

Conference port ID associated with this player.

Parameters
  • id: The file player ID.

pj_status_t pjsua_player_get_port(pjsua_player_id id, pjmedia_port **p_port)

Get the media port for the player or playlist.

Return

PJ_SUCCESS on success.

Parameters
  • id: The player ID.

  • p_port: The media port associated with the player.

pj_status_t pjsua_player_get_info(pjsua_player_id id, pjmedia_wav_player_info *info)

Get additional info about the file player. This operation is not valid for playlist.

Return

PJ_SUCCESS on success or the appropriate error code.

Parameters
  • port: The file player ID.

  • info: The info.

pj_ssize_t pjsua_player_get_pos(pjsua_player_id id)

Get playback position. This operation is not valid for playlist.

Return

The current playback position, in samples. On error, return the error code as negative value.

Parameters
  • id: The file player ID.

pj_status_t pjsua_player_set_pos(pjsua_player_id id, pj_uint32_t samples)

Set playback position. This operation is not valid for playlist.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • id: The file player ID.

  • samples: The playback position, in samples. Application can specify zero to re-start the playback.

pj_status_t pjsua_player_destroy(pjsua_player_id id)

Close the file of playlist, remove the player from the bridge, and free resources associated with the file player or playlist.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • id: The file player ID.

pj_status_t pjsua_recorder_create(const pj_str_t *filename, unsigned enc_type, void *enc_param, pj_ssize_t max_size, unsigned options, pjsua_recorder_id *p_id)

Create a file recorder, and automatically connect this recorder to the conference bridge. The recorder currently supports recording WAV file. The type of the recorder to use is determined by the extension of the file (e.g. “.wav”).

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • filename: Output file name. The function will determine the default format to be used based on the file extension. Currently “.wav” is supported on all platforms.

  • enc_type: Optionally specify the type of encoder to be used to compress the media, if the file can support different encodings. This value must be zero for now.

  • enc_param: Optionally specify codec specific parameter to be passed to the file writer. For .WAV recorder, this value must be NULL.

  • max_size: Maximum file size. Specify zero or -1 to remove size limitation. This value must be zero or -1 for now.

  • options: Optional options.

  • p_id: Pointer to receive the recorder instance.

pjsua_conf_port_id pjsua_recorder_get_conf_port(pjsua_recorder_id id)

Get conference port associated with recorder.

Return

Conference port ID associated with this recorder.

Parameters
  • id: The recorder ID.

pj_status_t pjsua_recorder_get_port(pjsua_recorder_id id, pjmedia_port **p_port)

Get the media port for the recorder.

Return

PJ_SUCCESS on success.

Parameters
  • id: The recorder ID.

  • p_port: The media port associated with the recorder.

pj_status_t pjsua_recorder_destroy(pjsua_recorder_id id)

Destroy recorder (this will complete recording).

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • id: The recorder ID.

pj_status_t pjsua_enum_aud_devs(pjmedia_aud_dev_info info[], unsigned *count)

Enum all audio devices installed in the system.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • info: Array of info to be initialized.

  • count: On input, specifies max elements in the array. On return, it contains actual number of elements that have been initialized.

pj_status_t pjsua_enum_snd_devs(pjmedia_snd_dev_info info[], unsigned *count)

Enum all sound devices installed in the system (old API).

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • info: Array of info to be initialized.

  • count: On input, specifies max elements in the array. On return, it contains actual number of elements that have been initialized.

pj_status_t pjsua_get_snd_dev(int *capture_dev, int *playback_dev)

Get currently active sound devices. If sound devices has not been created (for example when pjsua_start() is not called), it is possible that the function returns PJ_SUCCESS with -1 as device IDs. See also pjsua_snd_dev_id constants.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • capture_dev: On return it will be filled with device ID of the capture device.

  • playback_dev: On return it will be filled with device ID of the device ID of the playback device.

pj_status_t pjsua_set_snd_dev(int capture_dev, int playback_dev)

Select or change sound device. Application may call this function at any time to replace current sound device.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • capture_dev: Device ID of the capture device.

  • playback_dev: Device ID of the playback device.

pj_status_t pjsua_set_snd_dev2(pjsua_snd_dev_param *snd_param)

Select or change sound device according to the specified param.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • snd_param: Sound device param.

pj_status_t pjsua_set_null_snd_dev(void)

Set pjsua to use null sound device. The null sound device only provides the timing needed by the conference bridge, and will not interract with any hardware.

Return

PJ_SUCCESS on success, or the appropriate error code.

pjmedia_port *pjsua_set_no_snd_dev(void)

Disconnect the main conference bridge from any sound devices, and let application connect the bridge to it’s own sound device/master port.

Return

The port interface of the conference bridge, so that application can connect this to it’s own sound device or master port.

pj_status_t pjsua_set_ec(unsigned tail_ms, unsigned options)

Change the echo cancellation settings.

The behavior of this function depends on whether the sound device is currently active, and if it is, whether device or software AEC is being used.

If the sound device is currently active, and if the device supports AEC, this function will forward the change request to the device and it will be up to the device on whether support the request. If software AEC is being used (the software EC will be used if the device does not support AEC), this function will change the software EC settings. In all cases, the setting will be saved for future opening of the sound device.

If the sound device is not currently active, this will only change the default AEC settings and the setting will be applied next time the sound device is opened.

Return

PJ_SUCCESS on success.

Parameters
  • tail_ms: The tail length, in miliseconds. Set to zero to disable AEC.

  • options: Options to be passed to pjmedia_echo_create(). Normally the value should be zero.

pj_status_t pjsua_get_ec_tail(unsigned *p_tail_ms)

Get current echo canceller tail length.

Return

PJ_SUCCESS on success.

Parameters
  • p_tail_ms: Pointer to receive the tail length, in miliseconds. If AEC is disabled, the value will be zero.

pj_status_t pjsua_get_ec_stat(pjmedia_echo_stat *p_stat)

Get echo canceller statistics.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • p_stat: Pointer to receive the stat.

pj_bool_t pjsua_snd_is_active(void)

Check whether the sound device is currently active. The sound device may be inactive if the application has set the auto close feature to non-zero (the snd_auto_close_time setting in pjsua_media_config), or if null sound device or no sound device has been configured via the pjsua_set_no_snd_dev() function.

pj_status_t pjsua_snd_set_setting(pjmedia_aud_dev_cap cap, const void *pval, pj_bool_t keep)

Configure sound device setting to the sound device being used. If sound device is currently active, the function will forward the setting to the sound device instance to be applied immediately, if it supports it.

The setting will be saved for future opening of the sound device, if the “keep” argument is set to non-zero. If the sound device is currently inactive, and the “keep” argument is false, this function will return error.

Note that in case the setting is kept for future use, it will be applied to any devices, even when application has changed the sound device to be used.

Note also that the echo cancellation setting should be set with pjsua_set_ec() API instead.

See also pjmedia_aud_stream_set_cap() for more information about setting an audio device capability.

Return

PJ_SUCCESS on success or the appropriate error code.

Parameters
  • cap: The sound device setting to change.

  • pval: Pointer to value. Please see pjmedia_aud_dev_cap documentation about the type of value to be supplied for each setting.

  • keep: Specify whether the setting is to be kept for future use.

pj_status_t pjsua_snd_get_setting(pjmedia_aud_dev_cap cap, void *pval)

Retrieve a sound device setting. If sound device is currently active, the function will forward the request to the sound device. If sound device is currently inactive, and if application had previously set the setting and mark the setting as kept, then that setting will be returned. Otherwise, this function will return error.

Note that echo cancellation settings should be retrieved with pjsua_get_ec_tail() API instead.

Return

PJ_SUCCESS on success or the appropriate error code.

Parameters
  • cap: The sound device setting to retrieve.

  • pval: Pointer to receive the value. Please see pjmedia_aud_dev_cap documentation about the type of value to be supplied for each setting.

pj_status_t pjsua_ext_snd_dev_create(pjmedia_snd_port_param *param, pjsua_ext_snd_dev **p_snd)

Create an extra sound device and register it to conference bridge.

Return

PJ_SUCCESS on success or the appropriate error code.

Parameters
  • snd_param: Sound device port param. Currently this only supports mono channel, so channel count must be set to 1.

  • p_snd: The extra sound device instance.

pj_status_t pjsua_ext_snd_dev_destroy(pjsua_ext_snd_dev *snd)

Destroy an extra sound device and unregister it from conference bridge.

Return

PJ_SUCCESS on success or the appropriate error code.

Parameters
  • p_snd: The extra sound device instance.

pjmedia_snd_port *pjsua_ext_snd_dev_get_snd_port(pjsua_ext_snd_dev *snd)

Get sound port instance of an extra sound device.

Return

The sound port instance.

Parameters
  • snd: The extra sound device instance.

pjsua_conf_port_id pjsua_ext_snd_dev_get_conf_port(pjsua_ext_snd_dev *snd)

Get conference port ID of an extra sound device.

Return

The conference port ID.

Parameters
  • snd: The extra sound device instance.

pj_status_t pjsua_enum_codecs(pjsua_codec_info id[], unsigned *count)

Enum all supported codecs in the system.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • id: Array of ID to be initialized.

  • count: On input, specifies max elements in the array. On return, it contains actual number of elements that have been initialized.

pj_status_t pjsua_codec_set_priority(const pj_str_t *codec_id, pj_uint8_t priority)

Change codec priority.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • codec_id: Codec ID, which is a string that uniquely identify the codec (such as “speex/8000”). Please see pjsua manual or pjmedia codec reference for details.

  • priority: Codec priority, 0-255, where zero means to disable the codec.

pj_status_t pjsua_codec_get_param(const pj_str_t *codec_id, pjmedia_codec_param *param)

Get codec parameters.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • codec_id: Codec ID.

  • param: Structure to receive codec parameters.

pj_status_t pjsua_codec_set_param(const pj_str_t *codec_id, const pjmedia_codec_param *param)

Set codec parameters.

Return

PJ_SUCCESS on success, or the appropriate error code.

Parameters
  • codec_id: Codec ID.

  • param: Codec parameter to set. Set to NULL to reset codec parameter to library default settings.

struct pjsua_media_config
#include <pjsua.h>

This structure describes media configuration, which will be specified when calling pjsua_init(). Application MUST initialize this structure by calling pjsua_media_config_default().

Forward declaration

struct pjsua_codec_info
#include <pjsua.h>

This structure describes codec information, which can be retrieved by calling pjsua_enum_codecs().

struct pjsua_conf_port_info
#include <pjsua.h>

This structure describes information about a particular media port that has been registered into the conference bridge. Application can query this info by calling pjsua_conf_get_port_info().

struct pjsua_media_transport
#include <pjsua.h>

This structure holds information about custom media transport to be registered to pjsua.

struct pjsua_snd_dev_param
#include <pjsua.h>

This structure specifies the parameters to set the sound device. Use pjsua_snd_dev_param_default() to initialize this structure with default values.

struct pjsua_conf_connect_param
#include <pjsua.h>

This structure specifies the parameters for conference ports connection. Use pjsua_conf_connect_param_default() to initialize this structure with default values.