Membrane.RTC.Engine (Membrane RTC Engine v0.1.0-alpha.2) View Source

SFU engine implementation.

One SFU instance is responsible for managing one room in which all tracks of one peer are forwarded to all other peers.

The SFU engine works by sending and receiving messages. All messages are described below. To receive SFU messages you have to register your process so that SFU will know where to send its messages.

Registering for messages

Registration can be done by sending the message {register, pid} to the SFU instance, e.g.

send(sfu_pid, {:register, self()})

This will register your process to receive SFU messages. If your process implements GenServer behaviour then all messages will be handled by GenServer.handle_info/2, e.g.

@impl true
def handle_info({_sfu_engine, {:sfu_media_event, :broadcast, event}}, state) do
  for {_peer_id, pid} <- state.peer_channels, do: send(pid, {:media_event, event})
  {:noreply, state}
end

You can register multiple processes to receive messages from an SFU instance. In such a case each message will be sent to each registered process.

Media Events

The SFU engine needs to communicate with Membrane client libraries. This communication is done via Media Event messages. Media Events are blackbox messages that carry data important for the SFU engine and client libraries, but not for the user. Example Media Events are SDP offers, ICE candidates, and information about new peers.

An application is obligated to transport Media Events from an SFU instance to its client library, and vice versa.

When an SFU needs to send a message to a specific client, registered processes will receive {:sfu_media_event, to, event}, where to specifies the message destination. This can be either :broadcast, when the event should be sent to all peers, or peer_id when the messages should be sent to specified peer. The event is encoded in binary format, so it is ready to send without modification.

Feeding an SFU instance with Media Events from a client library can be done by sending the message {:media_event, from, event}. Assuming the user process is a GenServer, the Media Event can be received by GenServer.handle_info/2 and conveyed to the SFU engine in the following way:

@impl true
def handle_info({:media_event, _from, _event} = msg, state) do
  send(state.sfu_engine, msg)
  {:noreply, state}
end

What is important, Membrane SFU doesn't impose usage of any specific transport layer. You can e.g. use Phoenix and its channels. This can look like this:

@impl true
def handle_in("mediaEvent", %{"data" => event}, socket) do
  send(socket.assigns.room, {:media_event, socket.assigns.peer_id, event})

  {:noreply, socket}
end

Messages

Each message the SFU sends is a two-element tuple {sfu_pid, msg} where sfu_pid is the pid of the SFU instance that sent message, and msg can be any data.

Notice that thanks to presence of sfu_pid you can create multiple SFU instances.

Example SFU message:

{_sfu_pid, {:vad_notification, val, peer_id}}

SFU sends following messages

  • {:sfu_media_event, to, event} - a Media Event that should be transported to the client library. When from is :broadcast, the Media Event should be sent to all peers. When from is a peer_id, the Media Event should be sent to that specified peer.
  • {:vad_notification, val, peer_id} - sent when peer with id peer_id is speaking. VAD stands for Voice Activity Detection. When val is true marks start of speech whereas false marks end of speech.
  • {:new_peer, peer_id, metadata, track_metadata} - sent when a new peer tries to join to an SFU instance. metadata is any data passed by the client library while joining. track_metadata is a map where key is track id and value is its metadata defined in client library while adding a new track.
  • {:peer_left, peer_id} - sent when the peer with peer_id leaves an SFU instance

SFU receives following messages

  • {:register, pid} - register given pid for receiving SFU messages
  • {:unregister, pid} - unregister given pid from receiving SFU messages
  • {:media_event, from, event} - feed Media Event to SFU. from is id of peer that this Media event comes from.
  • {:accept_new_peer, peer_id} - accepts peer with id peer_id
  • {:deny_new_peer, peer_id} - denies peer with id peer_id
  • {:remove_peer, peer_id} - removes peer with id peer_id

Peer id

Peer ids must be assigned by application code. This is not done by the SFU engine or its client library. Ids can be assigned when a peer initializes its signaling channel.

Assuming we use a Phoenix channel as signaling layer:

def join("room:" <> room_id, _params, socket) do
  # ...
  peer_id = UUID.uuid4()
  {:ok, assign(socket, %{room_id: room_id, room: room, peer_id: peer_id})}
end

Link to this section Summary

Types

List of RTP extensions to use.

SFU network configuration options.

SFU configuration options.

A map pointing from encoding names to lists of packet filters that should be used for given encodings.

Functions

Changes playback state of pipeline to :playing.

Changes playback state to :prepared.

Changes playback state to :stopped.

Changes pipeline's playback state to :stopped and terminates its process.

Link to this section Types

Specs

extension_options_t() :: [{:vad, boolean()}]

List of RTP extensions to use.

At this moment only vad extension is supported. Enabling it will cause SFU sending {:vad_notification, val, endpoint_id} messages.

Specs

network_options_t() :: [
  stun_servers: [stun_server_t()],
  turn_servers: [turn_server_t()],
  dtls_pkey: binary(),
  dtls_cert: binary()
]

SFU network configuration options.

dtls_pkey and dtls_cert can be used e.g. when there are a lot of SFU instances and all of them need to use the same certificate and key.

Example configuration can look like this:

network_options: [
  stun_servers: [
    %{server_addr: "stun.l.google.com", server_port: 19_302}
  ]
]

Specs

options_t() :: [
  id: String.t(),
  extension_options: extension_options_t(),
  network_options: network_options_t(),
  packet_filters: %{required(encoding_name :: atom()) => [packet_filters_t()]}
]

SFU configuration options.

id is used by logger. If not provided it will be generated.

Specs

packet_filters_t() :: %{
  required(encoding_name :: atom()) => [
    Membrane.RTP.SessionBin.packet_filter_t()
  ]
}

A map pointing from encoding names to lists of packet filters that should be used for given encodings.

A sample usage would be to add silence discarder to OPUS tracks when VAD extension is enabled. It can greatly reduce CPU usage in rooms when there are a lot of people but only a few of them are actively speaking.

Specs

stun_server_t() :: ExLibnice.stun_server()

Specs

turn_server_t() :: ExLibnice.relay_info()

Link to this section Functions

Specs

get_registry_name() :: atom()

Specs

play(pid()) :: :ok

Changes playback state of pipeline to :playing.

Specs

prepare(pid()) :: :ok

Changes playback state to :prepared.

Link to this function

start(options, process_options)

View Source

Specs

start(options :: options_t(), process_options :: GenServer.options()) ::
  GenServer.on_start()
Link to this function

start_link(options, process_options)

View Source

Specs

start_link(options :: options_t(), process_options :: GenServer.options()) ::
  GenServer.on_start()

Specs

stop(pid()) :: :ok

Changes playback state to :stopped.

Link to this function

stop_and_terminate(pipeline, opts \\ [])

View Source

Specs

stop_and_terminate(pid(), Keyword.t()) :: :ok

Changes pipeline's playback state to :stopped and terminates its process.