View Source Membrane.RTC.Engine (Membrane RTC Engine v0.9.1)

RTC Engine implementation.

RTC Engine is an abstraction layer responsible for linking together different types of Endpoints. From the implementation point of view, RTC Engine is a Membrane.Pipeline.

messages

Messages

The RTC Engine works by sending messages which notify user logic about important events like "There is a new peer, would you like to to accept it?". To receive RTC Engine messages you have to register your process so that RTC Engine will know where to send them. All messages RTC Engine can emit are described in Membrane.RTC.Engine.Message docs.

registering-for-messages

Registering for messages

Registration can be done using register/2 e.g.

Engine.register(rtc_engine, self())

This will register your process to receive RTC Engine messages. If your process implements GenServer behavior then all messages can be handled by GenServer.handle_info/2, e.g.

@impl true
def handle_info(%Message.NewPeer{rtc_engine: rtc_engine, peer: peer}, state) do
  Engine.accept_peer(rtc_engine, peer.id)
  {:noreply, state}
end

You can register multiple processes to receive messages from an RTC Engine instance. In such a case each message will be sent to each registered process.

client-libraries

Client Libraries

RTC Engine allows creating Client Libraries that can send and receive media tracks from it. The current version of RTC Engine ships with WebRTC Client Library which connects to the RTC Engine via WebRTC standard. Communication with Client Libraries is done using Media Events. Media Events are control messages which notify about e.g. new peer joining to the RTC Engine. When Client Library receives Media Event it can invoke some callbacks. In the case of WebRTC Client Library, these are e.g. onPeerJoined or onTrackAdded. When RTC Engine receives Media Event it can emit some messages e.g. Membrane.RTC.Engine.Message.NewPeer.t/0. More about Media Events can be read in subsequent sections. Below there is a figure showing the architecture of the RTC Engine working in conjunction with some Client Library.

    +--------------------------------- media events -----------------------------+
    |                                (signaling layer)                           |
    |                                                                            |
    |                                                                            |
+--------+                 +---------+             +--------+               +---------+
| user   | <-   media   -> | Client  |             |  RTC   | <- media   -> | user    |
| client |      events     | Library | <- media -> | Engine |    events     | backend |
| logic  | <- callbacks -  |         |             |        | - messages -> | logic   |
+--------+                 +---------+             +--------+               +---------+

media-events

Media Events

Media Events are blackbox messages that carry data important for the RTC Engine and its Client Library, but not for the user. There are two types of Media Events:

  • Internal Media Events - generic, protocol-agnostic Media Events sent by RTC Engine itself. Example Internal Media Events are peerJoined, peerLeft, tracksAdded or tracksRemoved.
  • Custom Media Events - they can be used to send custom data from Client Library to some Endpoint inside RTC Engine and vice versa. In the case of WebRTC Client Library, these are sdpOffer, sdpAnswer, or iceCandidate.

An application is obligated to transport Media Events from an RTC Engine instance to its Client Library, and vice versa.

When the RTC Engine needs to send a Media Event to a specific client, registered processes will receive Membrane.RTC.Engine.Message.MediaEvent.t/0 message with to field indicating where this Media Event should be sent to. This can be either :broadcast, when the event should be sent to all peers, or peer_id when the messages should be sent to the specified peer. The event is encoded in binary format, so it is ready to send without modification.

Feeding an RTC Engine instance with Media Events from a Client Library can be done using receive_media_event/2. Assuming the user process is a GenServer, the Media Event can be received by GenServer.handle_info/2 and conveyed to the RTC Engine in the following way:

@impl true
def handle_info({:media_event, from, event} = msg, state) do
  Engine.receive_media_event(state.rtc_engine, from, event)
  {:noreply, state}
end

What is important, Membrane RTC Engine doesn't impose usage of any specific transport layer for carrying Media Events through the network. You can e.g. use Phoenix and its channels. This can look like this:

@impl true
def handle_in("mediaEvent", %{"data" => event}, socket) do
  Engine.receive_media_event(socket.assigns.room, socket.assigns.peer_id, event)
  {:noreply, socket}
end

peers

Peers

Each peer represents some user that can possess some metadata. A Peer can be added in two ways:

  • by sending proper Media Event from a Client Library
  • using add_peer/3

Adding a peer will cause RTC Engine to emit Media Event which will notify connected clients about new peer.

peer-id

Peer id

Peer ids must be assigned by application code. This is not done by the RTC Engine or its client library. Ids can be assigned when a peer initializes its signaling layer.

endpoints

Endpoints

Endpoints are Membrane.Bins able to publish their own tracks and subscribe to tracks from other Endpoints. One can think about Endpoint as an entity responsible for handling some specific task. An Endpoint can be added and removed using add_endpoint/3 and remove_endpoint/2 respectively.

There are two types of Endpoints:

  • Standalone Endpoints - they are in most cases spawned only once per RTC Engine instance and they are not associated with any peer.
  • Peer Endpoints - they are associated with some peer. Associating Endpoint with Peer will cause RTC Engine to send some Media Events to the Enpoint's Client Library e.g. one which indicates which tracks belong to which peer.

Currently RTC Engine ships with the implementation of two Endpoints:

  • Membrane.RTC.Engine.Endpoint.WebRTC which is responsible for establishing a connection with some WebRTC peer (mainly browser) and exchanging media with it. WebRTC Endpoint is a Peer Endpoint.
  • Membrane.RTC.Engine.Endpoint.HLS which is responsible for receiving media tracks from all other Endpoints and saving them to files by creating HLS playlists. HLS Endpoint is a Standalone Endpoint.

User can also implement custom Endpoints, see Custom Endpoints guide.

Link to this section Summary

Types

Membrane action that will generate Custom Media Event.

Endpoint configuration options.

RTC Engine configuration options.

Membrane action that will cause RTC Engine to publish some message to all other endpoints.

Types of messages that can be published to other Endpoints.

Subscription options.

Membrane action that will inform RTC Engine about track readiness.

Functions

Allows peer for joining to the RTC Engine

Adds endpoint to the RTC Engine

Adds peer to the RTC Engine

Deny peer from joining to the RTC Engine.

The same as deny_peer/2 but allows for passing any data that will be returned to the client.

Sends message to RTC Engine endpoint. If endpoint doesn't exist message is ignored

play(pid) deprecated

Changes playback state of pipeline to :playing.

prepare(pid) deprecated

Changes playback state to :prepared.

Sends Media Event to RTC Engine.

Registers process with pid who for receiving messages from RTC Engine

Removes endpoint from the RTC Engine

Removes peer from RTC Engine.

stop(pid) deprecated

Changes playback state to :stopped.

Changes pipeline's playback state to :stopped and terminates its process.

Subscribes an endpoint for a track.

Changes pipeline's playback state to :stopped and terminates its process.

Unregisters process with pid who from receiving messages from RTC Engine

Link to this section Types

Link to this type

custom_media_event_action_t()

View Source
@type custom_media_event_action_t() ::
  {:notify, {:custom_media_event, data :: binary()}}

Membrane action that will generate Custom Media Event.

@type endpoint_options_t() :: [
  endpoint_id: String.t(),
  peer_id: String.t(),
  node: node()
]

Endpoint configuration options.

  • peer_id - associate endpoint with existing peer
  • endpoint_id - assign endpoint id. If not provided it will be generated by RTC Engine. This option cannot be used together with peer_id. Endpoints associated with peers have the id peer_id.
  • node - node on which endpoint should be spawned. If not provided, current node is used.
@type options_t() :: [
  id: String.t(),
  trace_ctx: map(),
  telemetry_label: Membrane.TelemetryMetrics.label(),
  display_manager?: boolean()
]

RTC Engine configuration options.

  • id is used by logger. If not provided it will be generated.
  • trace_ctx is used by OpenTelemetry. All traces from this engine will be attached to this context. Example function from which you can get Otel Context is get_current/0 from OpenTelemetry.Ctx.
  • display_manager? - set to true if you want to limit number of tracks sent from Membrane.RTC.Engine.Endpoint.WebRTC to a browser.
@type publish_action_t() :: {:notify, {:publish, publish_message_t()}}

Membrane action that will cause RTC Engine to publish some message to all other endpoints.

@type publish_message_t() ::
  {:new_tracks, [Membrane.RTC.Engine.Track.t()]}
  | {:removed_tracks, [Membrane.RTC.Engine.Track.t()]}

Types of messages that can be published to other Endpoints.

@type subscription_opts_t() :: Keyword.t()

Subscription options.

Link to this type

track_ready_action_t()

View Source
@type track_ready_action_t() ::
  {:notify,
   {:track_ready, Membrane.RTC.Engine.Track.id(),
    Membrane.RTC.Engine.Track.encoding(), Membrane.RTC.Engine.Track.variant()}}

Membrane action that will inform RTC Engine about track readiness.

Link to this section Functions

Link to this function

accept_peer(pid, peer_id)

View Source
@spec accept_peer(pid :: pid(), peer_id :: String.t()) :: :ok

Allows peer for joining to the RTC Engine

Link to this function

add_endpoint(pid, endpoint, opts \\ [])

View Source
@spec add_endpoint(
  pid :: pid(),
  endpoint :: Membrane.ParentSpec.child_spec_t(),
  opts :: endpoint_options_t()
) :: :ok | :error

Adds endpoint to the RTC Engine

Returns :error when there are both peer_id and endpoint_id specified in opts. For more information refer to endpoint_options_t/0.

@spec add_peer(pid :: pid(), peer :: Membrane.RTC.Engine.Peer.t()) :: :ok

Adds peer to the RTC Engine

@spec deny_peer(pid :: pid(), peer_id :: String.t()) :: :ok

Deny peer from joining to the RTC Engine.

Link to this function

deny_peer(pid, peer_id, data)

View Source
@spec deny_peer(pid :: pid(), peer_id :: String.t(), [{:data, any()}]) :: :ok

The same as deny_peer/2 but allows for passing any data that will be returned to the client.

This can be used for passing reason of peer refusal.

@spec get_registry_name() :: atom()
Link to this function

message_endpoint(rtc_engine, endpoint_id, message)

View Source
@spec message_endpoint(
  rtc_engine :: pid(),
  endpoint_id :: String.t(),
  message :: any()
) :: :ok

Sends message to RTC Engine endpoint. If endpoint doesn't exist message is ignored

This function is deprecated. use pipeline's :playback action instead.
@spec play(pid()) :: :ok

Changes playback state of pipeline to :playing.

This function is deprecated. use pipeline's :playback action instead.
@spec prepare(pid()) :: :ok

Changes playback state to :prepared.

Link to this function

receive_media_event(rtc_engine, media_event)

View Source
@spec receive_media_event(
  rtc_engine :: pid(),
  media_event :: {:media_event, pid(), any()}
) :: :ok

Sends Media Event to RTC Engine.

Link to this function

register(rtc_engine, who \\ self())

View Source
@spec register(rtc_engine :: pid(), who :: pid()) :: :ok

Registers process with pid who for receiving messages from RTC Engine

Link to this function

remove_endpoint(rtc_engine, id)

View Source
@spec remove_endpoint(
  pid :: pid(),
  id :: String.t()
) :: :ok

Removes endpoint from the RTC Engine

Link to this function

remove_peer(rtc_engine, peer_id, reason \\ nil)

View Source
@spec remove_peer(rtc_engine :: pid(), peer_id :: any(), reason :: String.t() | nil) ::
  :ok

Removes peer from RTC Engine.

If reason is other than nil, RTC Engine will inform client library about peer removal with passed reason.

Link to this function

start(options, process_options)

View Source
@spec start(options :: options_t(), process_options :: GenServer.options()) ::
  GenServer.on_start()
Link to this function

start_link(options, process_options)

View Source
@spec start_link(options :: options_t(), process_options :: GenServer.options()) ::
  GenServer.on_start()
This function is deprecated. use pipeline's :playback action instead.
@spec stop(pid()) :: :ok

Changes playback state to :stopped.

Link to this function

stop_and_terminate(pipeline, opts \\ [])

View Source
This function is deprecated. use terminate/2 instead.
@spec stop_and_terminate(pid(), Keyword.t()) :: :ok

Changes pipeline's playback state to :stopped and terminates its process.

Link to this function

subscribe(rtc_engine, endpoint_id, track_id, opts \\ [])

View Source
@spec subscribe(
  rtc_engine :: pid(),
  endpoint_id :: String.t(),
  track_id :: Membrane.RTC.Engine.Track.id(),
  opts :: subscription_opts_t()
) :: :ok | {:error, :timeout | :invalid_track_id}

Subscribes an endpoint for a track.

The endpoint will be notified about track readiness in Membrane.Bin.handle_pad_added/3 callback. endpoint_id is the id of the endpoint, which wants to subscribe to the track.

Link to this function

terminate(pipeline, opts \\ [])

View Source
@spec terminate(pid(), Keyword.t()) :: :ok

Changes pipeline's playback state to :stopped and terminates its process.

Link to this function

unregister(rtc_engine, who \\ self())

View Source
@spec unregister(rtc_engine :: pid(), who :: pid()) :: :ok

Unregisters process with pid who from receiving messages from RTC Engine