BB.Camera behaviour (bb_camera v0.1.1)

Copy Markdown View Source

Behaviour and API for cameras in the BB framework.

A camera is an ordinary BB.Sensor that happens to produce rather more data than a sensor usually does. It is declared in the robot DSL like any other sensor, supervised like any other sensor, and bb itself knows nothing about cameras.

topology do
  link :head do
    sensor :eye, {BB.Camera.Synthetic, width: 640, height: 480}
  end
end

Frames do not go on PubSub

Everything else in bb publishes its readings and lets subscribers sort it out. Cameras cannot: a 1080p MJPEG stream is tens of megabytes a second, and a fan-out registry with no backpressure turns one slow subscriber into an unbounded mailbox. So BB.PubSub carries camera lifecycle only - capture started and ended - and frames travel over the bounded interface BB.Camera.Hub serves.

The split is not an optimisation. It is what lets a robot keep running when a consumer stalls.

Consuming a camera

Two shapes over the same mechanism. Pull, which suits anything with its own loop - an HTTP route writing chunks, say:

{:ok, session} = BB.Camera.Media.open(MyRobot, :eye, format: :jpeg)
{:ok, frame} = BB.Camera.Media.next(session, 2_000)
:ok = BB.Camera.Media.close(session)

And demand, which suits anything with its own notion of backpressure - a Membrane pipeline, or a GenServer that wants frames as messages:

{:ok, sub} = BB.Camera.subscribe(MyRobot, :eye)
:ok = BB.Camera.request(sub)
receive do
  {:bb_camera, ref, %BB.Camera.Frame{} = frame} when ref == sub.ref -> frame
end

In both, a request yields exactly one frame, strictly newer than the last one you were given. A consumer that asks in a loop therefore runs at the camera's frame rate; one that asks slowly gets the latest frame each time and misses the ones in between, which for independently decodable frames is lossless for what survives. That is also why this package refuses interframe codecs outright - see BB.Camera.Format.

Writing a driver

One module, use BB.Sensor and use BB.Camera. One callback of your own, then two messages from the hub to answer. BB.Camera.Synthetic is a complete worked example; the short version is:

defmodule MyCamera do
  use BB.Sensor, options_schema: [...]
  use BB.Camera

  @impl BB.Camera
  def stream_info(opts) do
    BB.Camera.StreamInfo.new(format: :jpeg, width: opts[:width], height: opts[:height])
  end

  @impl BB.Sensor
  def init(opts) do
    {:ok, info} = stream_info(opts)
    {:ok, hub} = BB.Camera.Hub.start_link(bb: opts[:bb], stream_info: info)
    {:ok, %{bb: opts[:bb], hub: hub}}
  end

  @impl BB.Sensor
  def handle_info({:bb_camera, :start_capture}, state), do: ...
  def handle_info({:bb_camera, :stop_capture}, state), do: ...
end

A camera is configured like any other component: through its BB.Sensor.options_schema/0, which the robot's DSL can fill from parameters with param/1. Changing one of those parameters at runtime re-validates the options and calls BB.Sensor.handle_options/2, which is where a driver reconfigures the device.

Nothing in this interface assumes where frames come from. A driver reading a device through a subprocess, one calling a platform capture API from a NIF, and one synthesising frames all meet it identically.

Summary

Callbacks

What this camera will produce, derived from its declared options alone.

Functions

Withdraw an outstanding request.

Every camera robot declares, whether or not it is running.

The hub serving a camera, if it's running.

Whether frames in format decode without reference to each other.

Ask for one frame.

What a camera is producing.

Become a consumer of a camera's capture.

Stop consuming. Capture stops when the last subscription goes.

Callbacks

stream_info(opts)

@callback stream_info(opts :: keyword()) ::
  {:ok, BB.Camera.StreamInfo.t()} | {:error, struct()}

What this camera will produce, derived from its declared options alone.

This must be pure. It is called during discovery, against robots that are not running, with the options exactly as written in the DSL - parameter references unresolved, hardware unopened, possibly absent. Describe what was asked for.

What the camera turns out to actually produce is reported at runtime through BB.Camera.Hub.capture_started/2 or BB.Camera.Hub.update_stream_info/2, and that value is what consumers see once it exists.

Return {:error, %BB.Error.Invalid.Camera.UnknownStreamInfo{}} when the options genuinely cannot imply a format; discovery then reports stream_info: nil rather than inventing one.

Functions

cancel_request(subscription)

@spec cancel_request(BB.Camera.Subscription.t()) :: :ok | {:error, struct()}

Withdraw an outstanding request.

Leaves the subscription in place. Use it when you've stopped waiting - after a timeout, say - so the hub isn't holding demand nothing will collect.

discover(robot, opts \\ [])

@spec discover(module(), keyword()) :: [BB.Camera.Info.t()]

Every camera robot declares, whether or not it is running.

Cameras that are declared but have no live hub are reported with status: :declared rather than omitted.

Options

  • :status - keep only cameras in this state (:capturing, :running or :declared). Defaults to all of them.

hub(robot, camera)

@spec hub(module(), atom()) :: {:ok, pid()} | {:error, struct()}

The hub serving a camera, if it's running.

independent_frames?(format)

@spec independent_frames?(term()) :: boolean()

Whether frames in format decode without reference to each other.

The predicate this package's format support is gated on. See BB.Camera.Format.

request(subscription)

@spec request(BB.Camera.Subscription.t()) :: :ok | {:error, struct()}

Ask for one frame.

You will be sent {:bb_camera, subscription.ref, %BB.Camera.Frame{}} once a frame strictly newer than your last one exists - immediately if one already does.

At most one request is outstanding at a time; asking again before delivery changes nothing. Demand does not accumulate, so a consumer that stops asking simply stops receiving, and the camera's memory use doesn't depend on how long it takes to come back.

These also arrive on a subscription, regardless of outstanding demand:

  • {:bb_camera, ref, {:stream_info, info}} immediately before the first frame a changed format describes.
  • {:bb_camera, ref, :end_of_capture} when capture stopped cleanly.
  • {:bb_camera, ref, {:capture_failed, error}} when it failed.

stream_info(robot, camera)

@spec stream_info(module(), atom()) ::
  {:ok, BB.Camera.StreamInfo.t()} | {:error, struct()}

What a camera is producing.

The live format when the camera is running, the declared one when it isn't.

subscribe(robot, camera, opts \\ [])

@spec subscribe(module(), atom(), keyword()) ::
  {:ok, BB.Camera.Subscription.t()} | {:error, struct()}

Become a consumer of a camera's capture.

Subscribing delivers nothing by itself - call request/1 for each frame you want. It does start capture if nobody else was watching, and the camera stops capturing when the last subscription goes.

Options

  • :format - assert the camera produces this format, failing loudly if it doesn't. A consumer that can only handle JPEG should say so here rather than discover it from the bytes.
  • :owner - the process frames are sent to, defaulting to the caller. The hub monitors it and unsubscribes automatically when it dies, so a crashed consumer releases the capture rather than pinning it.

The returned subscription's monitor watches the hub and belongs to the calling process, which is not necessarily :owner. Subscribing on one process's behalf from another is supported - the frames go where you said - but the owner won't learn of a dead camera from a monitor it doesn't hold. Subscribe from the process that will consume unless you have a reason not to.

unsubscribe(subscription)

@spec unsubscribe(BB.Camera.Subscription.t()) :: :ok

Stop consuming. Capture stops when the last subscription goes.