BB.Camera.FFmpeg.Capture (bb_camera_ffmpeg v0.1.0)

Copy Markdown View Source

One running capture: the ffmpeg process, the framer, and the bounds.

A camera driver starts one of these when its hub asks for capture and stops it when the hub says the last consumer has gone. Consumers never reach it — they talk to BB.Camera.Hub, which refcounts them, so two browser tabs on the same camera produce one of these, not two.

It is a process of its own rather than part of the driver so that the driver stays answerable while a frame delivery is in flight. BB.Camera.Hub.deliver/2 is a synchronous call by design, and a driver blocked inside one would not be reading the message asking it to stop.

The driver supplies the ffmpeg argument vector; this module neither knows nor cares which input it names. It must include -nostdin and write MJPEG to pipe:1 — see BB.Camera.FFmpeg.Backend.MuonTrap.

Bounding ingress

ffmpeg runs under MuonTrap, which forwards at most :stdio_window bytes of output before this process acknowledges them. Each read is acknowledged as it is taken from the mailbox, so the mailbox never holds more than one window, however fast ffmpeg writes and however slowly this process is scheduled. When this process falls a whole window behind, ffmpeg blocks on its write and the backlog stays outside the VM.

Within that bound, frames are shed rather than delivered while more than :drop_queue_len reads are waiting: the hub keeps only the newest frame, so a frame that would be superseded before anyone could see it is not worth a blocking call. The queue drains while frames are shed, and delivery resumes below the threshold. Within a single read, likewise, only the newest frame is delivered.

:max_frame_bytes bounds the framer's buffer, so memory held here is at most one window plus one frame.

Confirming capture

ffmpeg starting says only that ffmpeg exists. The first whole image is the evidence that the camera opened, so it is held, the owner is sent {:ready, info}, and nothing reaches the sink until the owner calls go/1.

An ffmpeg that never produces a first frame is stopped after :start_timeout_ms, and the capture exits {:shutdown, {:no_frames, ms}}. Some inputs wait for a first frame forever: ffmpeg's AVFoundation input does when macOS withholds camera access, and ignores SIGTERM while it waits.

Stopping

stop/1 asks MuonTrap to stop ffmpeg, which escalates to SIGKILL after :stop_timeout_ms. The capture stays alive until the exit status arrives, because whatever starts the next capture needs the camera released first. If no exit status arrives even after that, it closes the port, which makes MuonTrap kill ffmpeg, and exits {:shutdown, :teardown_unconfirmed} so the owner knows the camera may still be held.

Timestamps, honestly

A frame is stamped when the read that completed it arrived in this process. That is not the moment the sensor was exposed. Between the two sit the camera's own pipeline, the OS's capture buffers, ffmpeg, MuonTrap, the pipe, and the BEAM's scheduler.

Nothing better is available on this path: the raw MJPEG stream ffmpeg writes is a bare concatenation of images with no container and therefore no timestamps at all. The stamps are monotonic and in arrival order, so inter-frame intervals are meaningful even though absolute times are offset.

Summary

Types

What capture tells its owner about, beyond the frames themselves.

Functions

Returns a specification to start this module under a supervisor.

Let frames start flowing to the sink.

Start a capture.

Counters for this capture: frames delivered, frames shed, bytes discarded.

Stop the capture, giving ffmpeg a chance to release the camera first.

Types

event()

@type event() ::
  {:exited, status :: integer()}
  | {:discarded, bytes :: pos_integer(), reason :: atom()}
  | {:dropped, frames :: pos_integer()}
  | {:sink_error, term()}
  | {:ready, BB.Camera.FFmpeg.Jpeg.info()}
  | {:no_frames, timeout_ms :: pos_integer()}

What capture tells its owner about, beyond the frames themselves.

frame()

@type frame() :: %{
  data: binary(),
  width: pos_integer(),
  height: pos_integer(),
  sequence: non_neg_integer(),
  monotonic_time: integer(),
  wall_time: integer(),
  metadata: map()
}

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

go(capture)

@spec go(GenServer.server()) :: :ok

Let frames start flowing to the sink.

A capture holds its frames until this is called, so that the hub cannot be handed a frame before the driver has confirmed to it that capture started — it would refuse one, and the frame would be lost for no reason.

Nothing is dropped in the meantime beyond what the newest-frame-wins policy would have dropped anyway: the most recent frame is held and delivered here.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Start a capture.

Options

  • :executable — path to ffmpeg. Required.
  • :args — ffmpeg's arguments, including -nostdin. Required.
  • :sink — a 1-arity function called with each frame/0, returning :ok or {:error, reason}. Called inline, so a synchronous sink is what bounds delivery. Required.
  • :notify — process to send event/0s to, tagged {:bb_camera_ffmpeg, capture_pid, event}. Default the starting process.
  • :sequence_start — first sequence number. A camera that restarts its capture must keep counting up, because the hub rejects a sequence that does not strictly increase. Default 0.
  • :max_frame_bytes — passed to BB.Camera.FFmpeg.Framer.
  • :drop_queue_len — shed frames while more reads than this are waiting. Default 8.
  • :stdio_window — bytes of output ffmpeg may have in flight before this process acknowledges them. Default 262144.
  • :start_timeout_ms — how long to wait for the first whole frame. Default 10000, which leaves time to answer an OS permission prompt.
  • :stop_timeout_ms — how long ffmpeg gets to exit on SIGTERM before it is sent SIGKILL. Default 2000.
  • :backend, :backend_opts — defaults BB.Camera.FFmpeg.Backend.MuonTrap and [].

stats(capture)

@spec stats(GenServer.server()) :: map()

Counters for this capture: frames delivered, frames shed, bytes discarded.

stop(capture)

@spec stop(GenServer.server()) :: :ok

Stop the capture, giving ffmpeg a chance to release the camera first.

Asynchronous, because the caller is the driver and the driver has a hub waiting to be told that capture has ended. Process.monitor/1 the capture first when you need to know it has finished.