Lyra Codec (experimental)

group PJMED_LYRA

Implementation of Lyra Codec.

Lyra is Google’s open-source neural speech codec (https://github.com/google/lyra), targeting low bit rates (3200, 6000, or 9200 bps) at speech-quality fidelity by running a TFLite model on every frame. Audio characteristics:

  • 16-bit PCM input.

  • 20 ms frame length.

  • Clock rates 8000 / 16000 / 32000 / 48000 Hz, individually gated by compile-time switches (see Building PJSIP with Lyra).

Once registered with pjmedia_codec_lyra_init() the codec is driven through the standard Codec Framework API; the rest of this section describes Lyra-specific settings, build integration, and the runtime model files Lyra needs.

Initial integration: #3949 .

Codec Settings

General codec settings (VAD, PLC) live in setting on pjmedia_codec_param. Lyra-specific settings (bit rate, model path) live in pjmedia_codec_lyra_config and are applied via:

pjmedia_codec_lyra_config lyra_cfg;
pjmedia_codec_lyra_get_config(&lyra_cfg);

lyra_cfg.bit_rate   = 6000;
lyra_cfg.model_path = pj_str("/var/data/lyra/model_coeffs");

pjmedia_codec_lyra_set_config(&lyra_cfg);

Defaults come from the compile-time PJMEDIA_CODEC_LYRA_DEFAULT_BIT_RATE and PJMEDIA_CODEC_LYRA_DEFAULT_MODEL_PATH macros, applied at pjmedia_codec_lyra_init() time.

Configuration Fields

The fields of pjmedia_codec_lyra_config:

  • bit_rate: The local endpoint’s decoder bit rate, advertised to the remote so it knows what rate to encode at. Valid values are 3200, 6000, and 9200 bps. The two endpoints may pick different decoder bit rates: each side reads the other’s advertised bitrate and encodes accordingly, so a 3200↔6000 asymmetric pairing is a normal configuration.

  • model_path: Folder containing the four Lyra model files. If the path is invalid (folder missing or files absent), codec creation fails. See Runtime Model Files for the file list and deployment implications.

SDP Format Parameters

Lyra is offered with a single fmtp parameter:

m=audio 4000 RTP/AVP 99
a=rtpmap:99 lyra/16000/1
a=fmtp:99 bitrate=3200

The payload type (here 99) is dynamic; the rtpmap encoding name is “lyra” and the clock rate is one of the enabled clock rates. The fmtp bitrate parameter carries the local endpoint’s decoder bit rate from pjmedia_codec_lyra_config.bit_rate.

Asymmetric bit rates work naturally — each side advertises its own decoder rate, the peer encodes at that rate. The bit rate cannot be renegotiated mid-stream; change it via SDP re-INVITE.

Building the Lyra Library

Lyra is an external dependency that must be built and installed before PJSIP can link against it. Authoritative build instructions live with the Lyra project:

https://github.com/google/lyra

Lyra uses Bazel (not autotools or CMake) and depends on Abseil, gulrak filesystem, and glog. PJSIP’s autoconf with-lyra flag expects the prefix to expose those headers under include/com_google_absl, include/gulrak_filesystem, and include/com_google_glog/src — see aconfigure.ac for the exact -I paths assembled into LYRA_CXXFLAGS. C++17 is required.

The Lyra project evolves frequently. The PJSIP integration was developed against Lyra v2 ( #3949 ); newer revisions may need adjustments to the configure / link flags.

Building PJSIP with Lyra

Three build paths are supported.

Autoconf: pass with-lyra=DIR to ./configure pointing at the Lyra installation prefix:

./configure --with-lyra=/opt/lyra

The configure script detects Lyra, defines PJMEDIA_HAS_LYRA_CODEC to 1, and sets PJMEDIA_CODEC_LYRA_DEFAULT_MODEL_PATH to <prefix>/model_coeffs. Use disable-lyra to opt out explicitly.

CMake: Lyra is discovered via the bundled cmake/FindLyra.cmake module, which uses pkg-config when available. The lookup is optional: if Lyra is not found, CMake disables PJMEDIA_WITH_LYRA_CODEC instead of failing configuration.

config_site.h fallback (Visual Studio, custom builds): set PJMEDIA_HAS_LYRA_CODEC to 1 in config_site.h and add Lyra’s include / lib paths to the project manually.

Compile-time clock-rate switches (see pjmedia-codec/config.h):

Each enabled clock rate registers a separate codec instance, so a peer can negotiate any of the enabled rates. Disable rates the application doesn’t need to shrink the codec list and avoid pointless negotiation rounds.

Runtime Model Files

Lyra requires four model files at runtime, located in the folder pjmedia_codec_lyra_config.model_path points at:

  • lyra_config.binarypb

  • lyragan.tflite

  • quantizer.tflite

  • soundstream_encoder.tflite

These ship with the Lyra source tree under model_coeffs/ and are consumed when a Lyra encoder or decoder instance is opened for a stream — there is no embedded fallback. A missing or invalid model_path therefore surfaces as a codec-open failure during call media setup, not at pjmedia_codec_lyra_init() time. For mobile / embedded deployments the files must be packaged in the application bundle and model_path set to the unpacked location.

Caveats

  • Experimental: restated for emphasis. No RFC, no IANA registration; the fmtp bitrate parameter is PJSIP-internal. Interop is essentially limited to other PJSIP peers running the same integration.

  • CPU cost: Lyra runs a TFLite model on every 20 ms frame. Per-call CPU is significantly higher than traditional codecs; profile on the target platform before shipping.

  • Speech only: Lyra is trained on speech and produces poor results on music or non-speech audio. Pair it with appropriate call routing.

  • Model file size: the four model files together total several megabytes; account for this in mobile bundle sizing.

Warning

Lyra is experimental in PJSIP. It is not a standardised codec — there is no RFC, no IANA-registered MIME type, and no formally specified SDP / RTP payload format. The fmtp:bitrate=N parameter PJSIP advertises (see SDP Format Parameters) is an internal convention, not an interoperable spec; expect interop limited to peers running the same PJSIP Lyra integration.

Warning

The “lyra” encoding name and the bitrate fmtp parameter are not registered with IANA and are not specified in any RFC. Interop with non-PJSIP Lyra integrations is not guaranteed.

Functions

pj_status_t pjmedia_codec_lyra_init(pjmedia_endpt *endpt)

Initialize and register lyra codec factory to pjmedia endpoint.

Parameters:

endpt – The pjmedia endpoint.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_codec_lyra_deinit(void)

Unregister lyra codec factory from pjmedia endpoint and deinitialize the lyra codec library.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_codec_lyra_get_config(pjmedia_codec_lyra_config *cfg)

Get the default Lyra configuration.

Parameters:

cfg – Lyra codec configuration.

Returns:

PJ_SUCCESS on success.

pj_status_t pjmedia_codec_lyra_set_config(const pjmedia_codec_lyra_config *cfg)

Set the default Lyra configuration.

Parameters:

cfg – Lyra codec configuration.

Returns:

PJ_SUCCESS on success.

struct pjmedia_codec_lyra_config
#include <lyra.h>

Lyra codec setting;

Public Members

unsigned bit_rate

The value represents the decoder bitrate requested by the receiver. Endpoints can be configured with different bitrates. For example, the local endpoint might be set to a bitrate of 3200, while the remote endpoint is set to 6000. In this scenario, the remote endpoint will send data at 3200 bitrate, while the local endpoint will send data at 6000 bitrate. Valid bitrate: 3200, 6000, 9200. By default it is set to PJMEDIA_CODEC_LYRA_DEFAULT_BIT_RATE.

pj_str_t model_path

Lyra required some additional (model) files, including lyra_config.binarypb , lyragan.tflite , quantizer.tflite and soundstream_encoder.tflite . This setting represents the folder containing the above files. The specified folder should contain these files. If an invalid folder is provided, the codec creation will fail.