Membrane.RTP.SessionBin (Membrane RTP plugin v0.8.1) View Source
Bin handling one RTP session, that may consist of multiple incoming and outgoing RTP streams.
Incoming streams
Incoming RTP streams can be connected via :rtp_input
pads. As each pad can provide multiple RTP streams,
they are distinguished basing on SSRC. Once a new stream is received, bin sends new_stream_notification_t/0
notification, meaning the parent should link Pad.ref(:output, ssrc)
pad to consuming components. The stream is
then depayloaded and forwarded via said pad.
Outgoing streams
To create an RTP stream, the source stream needs to be connected via Pad.ref(:input, ssrc)
pad and the sink -
via Pad.ref(:rtp_output, ssrc)
. At least one of :encoding
or :payload_type
options of :rtp_output
pad
must be provided too.
Payloaders and depayloaders
Payloaders are Membrane elements that transform stream so that it can be put into RTP packets, while depayloaders work the other way round. Different codecs require different payloaders and depayloaders.
By default SessionBin
will neither payload nor depayload incoming/outgoing streams, to do so a payloader/depayloader needs to be
passed via Pad.ref(:input, ssrc)
and Pad.ref(:output, ssrc)
pads options.
Payloading/Depayloading is necessary if we need to somehow transform the streams. If SessionBin
s main role is to route packets
then depayloading and payloading processes are redundant.
Payloaders and depayloaders can be found in membrane_rtp_X_plugin
packages, where X stands for codec name.
It's enough when such a plugin is added to dependencies. To determine which payloader/depayloader to use, one can use Membrane.RTP.PayloadFormatResolver
which given an encoding name should resolve to proper payloader/depayloader modules (if those previously have been registered via mentioned plugins).
For payloading and depayloading, SessionBin
uses respective bins Membrane.RTP.PayloaderBin
and Membrane.RTP.DepayloaderBin
which will be spawned once payloader/depayloader are passed explicitly via pads' options.
Important note
Payloaders and depayloaders are mostly needed when working with external media sources (in different formats than RTP). For applications such as an SFU it is not needed to either payload or depayload the RTP stream as we are always dealing with RTP format. In such a case, SessionBin will receive payloaded packets and work as a simple proxy just forwarding the packets (and decrypting them if necessary). Therefore it is possible to specify in newly added pads if payloaders/depayloaders should be used for the certain stream.
RTCP
RTCP packets for inbound stream can be provided either in-band or via a separate rtp_input
pad instance. Corresponding
receiver report packets will be sent back through rtcp_output
with the same id as rtp_input
for the RTP stream.
RTCP for outbound stream is not yet supported.
Bin options
Passed via struct Membrane.RTP.SessionBin.t/0
fmt_mapping
%{RTP.payload_type_t() => {RTP.encoding_name_t(), RTP.clock_rate_t()}}
Default value:
%{}
Mapping of the custom payload types ( > 95)rtcp_report_interval
Membrane.Time.t() | nil
Default value:
nil
Interval between sending subseqent RTCP receiver reports.receiver_ssrc_generator
(local_ssrcs :: [pos_integer], remote_ssrcs :: [pos_integer] -> ssrc :: pos_integer)
Default value:
&Membrane.RTP.SessionBin.generate_receiver_ssrc/2
Function generating receiver SSRCs. Default one generates random SSRC that is not inlocal_ssrcs
norremote_ssrcs
.secure?
boolean
Default value:
false
Specifies whether to use SRTP. Requires adding srtp dependency to work.srtp_policies
[ExLibSRTP.Policy.t()]
Default value:
[]
List of SRTP policies to use for decrypting packets. Used only whensecure?
is set totrue
. SeeExLibSRTP.Policy.t/0
for details.receiver_srtp_policies
[ExLibSRTP.Policy.t()] | nil
Default value:
nil
List of SRTP policies to use for encrypting receiver reports and other receiver RTCP packets. Used only whensecure?
is set totrue
. Defaults to the value ofsrtp_policies
. SeeExLibSRTP.Policy.t/0
for details.
Pads
:rtp_input
Availability | :on_request |
Caps | Membrane.RemoteStream , restrictions:<br /> type: :packetized ,<br /> content_format: one_of([nil, Membrane.RTP]) |
Demand unit | :buffers |
Direction | :input |
Mode | :pull |
Name | :rtp_input |
:input
Availability | :on_request |
Caps | :any |
Demand unit | :buffers |
Direction | :input |
Mode | :pull |
Name | :input |
Options:
payloader
module() | nil
Default value:
nil
Payloader's module that should be used for a media stream flowing through the pad. If set to nil then the payloading process gets skipped.
:rtcp_output
Availability | :on_request |
Caps | Membrane.RemoteStream , restrictions:<br /> type: :packetized ,<br /> content_format: Membrane.RTCP |
Demand unit | :buffers |
Direction | :output |
Mode | :pull |
Name | :rtcp_output |
:rtp_output
Availability | :on_request |
Caps | Membrane.RemoteStream , restrictions:<br /> type: :packetized ,<br /> content_format: Membrane.RTP |
Demand unit | :buffers |
Direction | :output |
Mode | :pull |
Name | :rtp_output |
Options:
payload_type
RTP.payload_type_t() | nil
Default value:
nil
Payload type of output stream. If not provided, determined from:encoding
.encoding
RTP.encoding_name_t() | nil
Default value:
nil
Encoding name of output stream. If not provided, determined from:payload_type
.clock_rate
integer() | nil
Default value:
nil
Clock rate to use. If not provided, determined from:payload_type
.
:output
Availability | :on_request |
Caps | :any |
Demand unit | :buffers |
Direction | :output |
Mode | :pull |
Name | :output |
Options:
depayloader
module() | nil
Default value:
nil
Depayloader's module that should be used for an outgoing media stream flowing through the pad. If set to nil then the depayloading process gets skipped.clock_rate
integer() | nil
Default value:
nil
Clock rate to use. If not provided, determined fromfmt_mapping
or defaults registered by proper plugins i.e.Membrane.RTP.X.Plugin
where X is the name of codec corresponding toencoding
.extensions
[extension_t()]
Default value:
[]
List of extensions. Currently:vad
is only supported.:vad
will turn on Voice Activity Detection mechanism firing appropriate notifications when needed. Should be set only for audio tracks. For more information refer toMembrane.RTP.VAD
module documentation. Extensions are applied in the same order as passed to the pad options.
packet_filters
[packet_filter_t()]
Default value:
[]
A list of filter elements that will be attached to the packets flow (added insideMembrane.RTP.StreamReceiveBin
). In case of SRTP filters are placed before the Decryptor. The order of provided elements is important as the filters are applied in FIFO order. A filter can be responsible e.g. for dropping silent audio packets when encountered VAD extension data in the packet header.rtcp_fir_interval
Membrane.Time.t() | nil
Default value:
1000000000
Interval between sending subseqent RTCP Full Intra Request packets.
Link to this section Summary
Types
A module that will be spawned and linked just before a newly created :output
pad representing
a single RTP stream.
Options for pad :input
Options for pad :output
A definition of a child that will be responsible for arbitrary packet filtering
inside Membrane.RTP.StreamReceiveBin
. Each filter should have just a single input and output
pad named accordingly.
Options for pad :rtp_output
Struct containing options for Membrane.RTP.SessionBin
Functions
Returns pads descriptions for Membrane.RTP.SessionBin
Returns description of options available for this module
Link to this section Types
Specs
extension_t() :: {extension_name :: atom(), extension_config :: Membrane.ParentSpec.child_spec_t()}
A module that will be spawned and linked just before a newly created :output
pad representing
a single RTP stream.
Given extension config must be a valid Membrane.Filter
.
An extension will be spawned inside the bin under {extension_name :: atom(), ssrc}
name.
Currently supported extensions are:
Example usage
{:vad, %Mebrane.RTP.VAD{time_window: 1_000_000}}
Specs
input_pad_opts_t() :: [{:payloader, module() | nil}]
Options for pad :input
Specs
new_stream_notification_t() :: Membrane.RTP.SSRCRouter.new_stream_notification_t()
Specs
output_pad_opts_t() :: [ depayloader: module() | nil, clock_rate: integer() | nil, extensions: [extension_t()], packet_filters: [packet_filter_t()], rtcp_fir_interval: Membrane.Time.t() | nil ]
Options for pad :output
Specs
packet_filter_t() :: {Membrane.Child.name_t(), Membrane.ParentSpec.child_spec_t()}
A definition of a child that will be responsible for arbitrary packet filtering
inside Membrane.RTP.StreamReceiveBin
. Each filter should have just a single input and output
pad named accordingly.
A filter can be responsible e.g. for dropping silent audio packets when encountered VAD extension data in the packet header.
Specs
rtp_output_pad_opts_t() :: [ payload_type: Membrane.RTP.payload_type_t() | nil, encoding: Membrane.RTP.encoding_name_t() | nil, clock_rate: integer() | nil ]
Options for pad :rtp_output
Specs
t() :: %Membrane.RTP.SessionBin{ fmt_mapping: %{ required(Membrane.RTP.payload_type_t()) => {Membrane.RTP.encoding_name_t(), Membrane.RTP.clock_rate_t()} }, receiver_srtp_policies: [ExLibSRTP.Policy.t()] | nil, receiver_ssrc_generator: (local_ssrcs :: [pos_integer()], remote_ssrcs :: [pos_integer()] -> ssrc :: pos_integer()), rtcp_report_interval: Membrane.Time.t() | nil, secure?: boolean(), srtp_policies: [ExLibSRTP.Policy.t()] }
Struct containing options for Membrane.RTP.SessionBin
Link to this section Functions
Specs
membrane_pads() :: [{Membrane.Pad.name_t(), Membrane.Pad.description_t()}]
Returns pads descriptions for Membrane.RTP.SessionBin
Specs
options() :: keyword()
Returns description of options available for this module