LiveExWebRTC.Publisher (live_ex_webrtc v0.8.0)

View Source

Component for sending audio and video via WebRTC from a browser to a Phoenix app (browser publishes).

It:

  • renders:
    • audio and video device selects
    • audio and video stream configs
    • stream recording toggle (with recordings enabled)
    • stream preview
    • transmission stats
  • on clicking "Start Streaming", creates WebRTC PeerConnection both on the client and server side
  • connects those two peer connections negotiatiing a single audio and video track
  • sends audio and video from selected devices to the live view process
  • publishes received audio and video packets to the configured PubSub
  • can optionally use the ExWebRTC Recorder to record the stream

When LiveExWebRTC.Player is used, audio and video packets are delivered automatically, assuming both components are configured with the same PubSub.

If LiveExWebRTC.Player is not used, you should use following topics and messages:

  • streams:audio:#{publisher_id}:#{audio_track_id} - for receiving audio packets
  • streams:video:#{publisher_id}:#{video_track_id}:#{layer} - for receiving video packets. The message is in form of {:live_ex_webrtc, :video, "l" | "m" | "h", ExRTP.Packet.t()} or {:live_ex_webrtc, :audio, ExRTP.Packet.t()}. Packets for non-simulcast video tracks are always sent with "h" identifier.
  • streams:info:#{publisher.id}" - for receiving information about publisher tracks and their layers. The message is in form of: {:live_ex_webrtc, :info | :bye, audio_track :: ExWebRTC.MediaStreamTrack.t(), video_track :: ExWebRTC.MediaStreamTrack.t()}.
  • publishers:#{publisher_id} for sending keyframe request. The message must be in form of {:live_ex_webrtc, :keyframe_req, "l" | "m" | "h"} E.g.
    PubSub.broadcast(LiveTwitch.PubSub, "publishers:my_publisher", {:live_ex_webrtc, :keyframe_req, "h"})

JavaScript Hook

Publisher live view requires JavaScript hook to be registered under Publisher name. The hook can be created using createPublisherHook function. For example:

import { createPublisherHook } from "live_ex_webrtc";
let Hooks = {};
const iceServers = [{ urls: "stun:stun.l.google.com:19302" }];
Hooks.Publisher = createPublisherHook(iceServers);
let liveSocket = new LiveSocket("/live", Socket, {
  // ...
  hooks: Hooks
});

Simulcast

Simulcast requires video codecs to be H264 (packetization mode 1) and/or VP8. E.g.

video_codecs = [
  %RTPCodecParameters{
    payload_type: 98,
    mime_type: "video/H264",
    clock_rate: 90_000,
    sdp_fmtp_line: %FMTP{
      pt: 98,
      level_asymmetry_allowed: true,
      packetization_mode: 1,
      profile_level_id: 0x42E01F
    }
  },
  %RTPCodecParameters{
    payload_type: 96,
    mime_type: "video/VP8",
    clock_rate: 90_000
  }
]

You can also use the shorthands for default H264 and VP8 codec parameters:

video_codecs = [:h264, :vp8]

Examples

defmodule LiveTwitchWeb.StreamerLive do
  use LiveTwitchWeb, :live_view

  alias LiveExWebRTC.Publisher

  @impl true
  def render(assigns) do
  ~H"""
  <Publisher.live_render socket={@socket} publisher={@publisher} />
  """
  end

  @impl true
  def mount(_params, _session, socket) do
    socket = Publisher.attach(socket, id: "publisher", pubsub: LiveTwitch.PubSub)
    {:ok, socket}
  end
end

Summary

Types

Called when WebRTC has connected.

Called when WebRTC has disconnected.

Called when recorder sends a message to the Publisher.

Called when recorder finishes stream recording.

t()

Functions

Attaches required hooks and creates t/0 struct.

Helper function for rendering Publisher live view.

Types

on_connected()

@type on_connected() :: (publisher_id :: String.t() -> any())

Called when WebRTC has connected.

on_disconnected()

@type on_disconnected() :: (publisher_id :: String.t() -> any())

Called when WebRTC has disconnected.

on_packet()

@type on_packet() :: (publisher_id :: String.t(),
                packet_type :: :audio | :video,
                layer :: nil | String.t(),
                packet :: ExRTP.Packet.t(),
                socket :: Phoenix.LiveView.Socket.t() ->
                  packet :: ExRTP.Packet.t())

on_recorder_message()

@type on_recorder_message() :: (publisher_id :: String.t(),
                          ExWebRTC.Recorder.message() ->
                            any())

Called when recorder sends a message to the Publisher.

For exact meaning of the second argument, refer to ExWebRTC.Recorder.message/0.

on_recording_finished()

@type on_recording_finished() :: (publisher_id :: String.t(),
                            ExWebRTC.Recorder.end_tracks_ok_result() ->
                              any())

Called when recorder finishes stream recording.

For exact meaning of the second argument, refer to ExWebRTC.Recorder.end_tracks_ok_result/0.

t()

@type t() :: struct()

Functions

attach(socket, opts)

Attaches required hooks and creates t/0 struct.

Created struct is saved in socket's assigns and has to be passed to LiveExWebRTC.Publisher.live_render/1.

Options:

live_render(assigns)

Helper function for rendering Publisher live view.

Attributes

  • socket (Phoenix.LiveView.Socket) (required) - Parent live view socket.
  • publisher (LiveExWebRTC.Publisher) (required) - Publisher struct. It is used to pass publisher id to the newly created live view via live view session. This data is then used to do a handshake between parent live view and child live view during which child live view receives the whole Publisher struct.