Membrane.RTC.Engine (Membrane RTC Engine v0.1.0-alpha.2) View Source
SFU engine implementation.
One SFU instance is responsible for managing one room in which all tracks of one peer are forwarded to all other peers.
The SFU engine works by sending and receiving messages. All messages are described below. To receive SFU messages you have to register your process so that SFU will know where to send its messages.
Registering for messages
Registration can be done by sending the message {register, pid}
to the SFU instance, e.g.
send(sfu_pid, {:register, self()})
This will register your process to receive SFU messages.
If your process implements GenServer
behaviour then all messages will be handled
by GenServer.handle_info/2
, e.g.
@impl true
def handle_info({_sfu_engine, {:sfu_media_event, :broadcast, event}}, state) do
for {_peer_id, pid} <- state.peer_channels, do: send(pid, {:media_event, event})
{:noreply, state}
end
You can register multiple processes to receive messages from an SFU instance. In such a case each message will be sent to each registered process.
Media Events
The SFU engine needs to communicate with Membrane client libraries.
This communication is done via Media Event
messages.
Media Events are blackbox messages that carry data important for the
SFU engine and client libraries, but not for the user. Example Media Events are
SDP offers, ICE candidates, and information about new peers.
An application is obligated to transport Media Events from an SFU instance to its client library, and vice versa.
When an SFU needs to send a message to a specific client, registered processes will
receive {:sfu_media_event, to, event}
, where to
specifies the message destination.
This can be either :broadcast
, when the event should be sent to all peers, or peer_id
when the messages should be sent to specified peer. The event
is encoded in binary format,
so it is ready to send without modification.
Feeding an SFU instance with Media Events from a client library can be done by sending the
message {:media_event, from, event}
. Assuming the user process is a GenServer, the
Media Event can be received by GenServer.handle_info/2
and conveyed to the SFU engine in
the following way:
@impl true
def handle_info({:media_event, _from, _event} = msg, state) do
send(state.sfu_engine, msg)
{:noreply, state}
end
What is important, Membrane SFU doesn't impose usage of any specific transport layer. You can e.g. use Phoenix and its channels. This can look like this:
@impl true
def handle_in("mediaEvent", %{"data" => event}, socket) do
send(socket.assigns.room, {:media_event, socket.assigns.peer_id, event})
{:noreply, socket}
end
Messages
Each message the SFU sends is a two-element tuple {sfu_pid, msg}
where
sfu_pid
is the pid of the SFU instance that sent message, and msg
can be any data.
Notice that thanks to presence of sfu_pid
you can create multiple SFU instances.
Example SFU message:
{_sfu_pid, {:vad_notification, val, peer_id}}
SFU sends following messages
{:sfu_media_event, to, event}
- a Media Event that should be transported to the client library. Whenfrom
is:broadcast
, the Media Event should be sent to all peers. Whenfrom
is apeer_id
, the Media Event should be sent to that specified peer.{:vad_notification, val, peer_id}
- sent when peer with idpeer_id
is speaking.VAD
stands forVoice Activity Detection
. Whenval
istrue
marks start of speech whereasfalse
marks end of speech.{:new_peer, peer_id, metadata, track_metadata}
- sent when a new peer tries to join to an SFU instance.metadata
is any data passed by the client library while joining.track_metadata
is a map where key is track id and value is its metadata defined in client library while adding a new track.{:peer_left, peer_id}
- sent when the peer withpeer_id
leaves an SFU instance
SFU receives following messages
{:register, pid}
- register givenpid
for receiving SFU messages{:unregister, pid}
- unregister givenpid
from receiving SFU messages{:media_event, from, event}
- feed Media Event to SFU.from
is id of peer that this Media event comes from.{:accept_new_peer, peer_id}
- accepts peer with idpeer_id
{:deny_new_peer, peer_id}
- denies peer with idpeer_id
{:remove_peer, peer_id}
- removes peer with idpeer_id
Peer id
Peer ids must be assigned by application code. This is not done by the SFU engine or its client library. Ids can be assigned when a peer initializes its signaling channel.
Assuming we use a Phoenix channel as signaling layer:
def join("room:" <> room_id, _params, socket) do
# ...
peer_id = UUID.uuid4()
{:ok, assign(socket, %{room_id: room_id, room: room, peer_id: peer_id})}
end
Link to this section Summary
Types
List of RTP extensions to use.
SFU network configuration options.
SFU configuration options.
A map pointing from encoding names to lists of packet filters that should be used for given encodings.
Functions
Changes playback state of pipeline to :playing
.
Changes playback state to :prepared
.
Changes playback state to :stopped
.
Changes pipeline's playback state to :stopped
and terminates its process.
Link to this section Types
Specs
extension_options_t() :: [{:vad, boolean()}]
List of RTP extensions to use.
At this moment only vad
extension is supported.
Enabling it will cause SFU sending {:vad_notification, val, endpoint_id}
messages.
Specs
network_options_t() :: [ stun_servers: [stun_server_t()], turn_servers: [turn_server_t()], dtls_pkey: binary(), dtls_cert: binary() ]
SFU network configuration options.
dtls_pkey
and dtls_cert
can be used e.g. when there are a lot of SFU instances
and all of them need to use the same certificate and key.
Example configuration can look like this:
network_options: [
stun_servers: [
%{server_addr: "stun.l.google.com", server_port: 19_302}
]
]
Specs
options_t() :: [ id: String.t(), extension_options: extension_options_t(), network_options: network_options_t(), packet_filters: %{required(encoding_name :: atom()) => [packet_filters_t()]} ]
SFU configuration options.
id
is used by logger. If not provided it will be generated.
Specs
packet_filters_t() :: %{ required(encoding_name :: atom()) => [ Membrane.RTP.SessionBin.packet_filter_t() ] }
A map pointing from encoding names to lists of packet filters that should be used for given encodings.
A sample usage would be to add silence discarder to OPUS tracks when VAD extension is enabled. It can greatly reduce CPU usage in rooms when there are a lot of people but only a few of them are actively speaking.
Specs
stun_server_t() :: ExLibnice.stun_server()
Specs
turn_server_t() :: ExLibnice.relay_info()
Link to this section Functions
Specs
get_registry_name() :: atom()
Specs
play(pid()) :: :ok
Changes playback state of pipeline to :playing
.
Specs
prepare(pid()) :: :ok
Changes playback state to :prepared
.
Specs
start(options :: options_t(), process_options :: GenServer.options()) :: GenServer.on_start()
Specs
start_link(options :: options_t(), process_options :: GenServer.options()) :: GenServer.on_start()
Specs
stop(pid()) :: :ok
Changes playback state to :stopped
.
Specs
Changes pipeline's playback state to :stopped
and terminates its process.