ALLM.Audio (allm v0.6.1)

Copy Markdown View Source

A serializable audio value, used as transcription input and as speech synthesis output. Layer A serializable data.

One value type serves both directions. ALLM.TranscriptionRequest carries an %Audio{} in :audio, and ALLM.SpeechResponse returns the synthesized bytes as %Audio{source: {:binary, bytes}}.

iex> audio = ALLM.Audio.from_binary(<<1, 2, 3>>, "audio/wav")
iex> ALLM.Audio.size(audio)
{:ok, 3}

Source variants

  • {:binary, bytes}: raw audio bytes plus an explicit :mime_type, via from_binary/2.
  • {:base64, encoded}: standard base64-encoded bytes plus an explicit :mime_type, via from_base64/2. The string is stored as given and is not checked until to_binary/1 or size/1 decodes it.
  • {:file, path}: a local filesystem path, via from_file/1. Building one does no I/O. The file is read only by to_binary/1, and size/1 only stats it.

There is no URL variant: no bundled provider accepts a URL for audio input.

MIME types

from_file/1 infers :mime_type from the lowercased extension: .mp3, .mpeg, .mpga → audio/mpeg; .mp4, .m4a → audio/mp4; .wav → audio/wav; .webm → audio/webm; .ogg, .oga → audio/ogg; .flac → audio/flac; .aac → audio/aac; .opus → audio/opus; .aiff → audio/aiff. Any other extension leaves it nil, and the adapter decides what to do with an unknown type.

Construction

Build values with the from_* functions. There is no new/1. :source is enforced, so struct!(ALLM.Audio, []) raises ArgumentError.

Inspection

inspect/1 prints a payload as its size, never its bytes, so a multi-megabyte clip in an error message, a test failure diff, a log line or a telemetry handler costs one short line:

iex> inspect(ALLM.Audio.from_binary(<<0, 0, 0, 0>>, "audio/wav"))
~s(#ALLM.Audio<source: {:binary, <<4 bytes>>}, mime_type: "audio/wav", metadata: %{}>)

Serializability

ETF round-trip preserves every source variant verbatim. JSON round-trip through ALLM.Serializer writes :source as %{"type" => "<kind>", "value" => <value>}, and the {:binary, _} variant base64-encodes its bytes so that non-UTF-8 audio survives as JSON text. A "binary" source whose value is not valid base64 decodes to the field error {[:source], :invalid_base64}. An unknown source type decodes to {:_unknown, :atom_decode_failed}.

:metadata is caller-owned. Use string keys when it will round-trip through JSON: atom keys come back as strings.

Summary

Types

source()
t()

Functions

from_base64(encoded, mime_type)

Build an %Audio{} from a base64-encoded string plus an explicit MIME type.

from_binary(bytes, mime_type)

Build an %Audio{} from raw bytes plus an explicit MIME type.

from_file(path)

Build an %Audio{} from a local filesystem path.

size(audio)

The audio payload's size in bytes, without reading a file.

to_binary(audio)

Resolve the audio's :source to raw bytes.

Types

source()

@type source() :: {:binary, binary()} | {:base64, String.t()} | {:file, Path.t()}

t()

@type t() :: %ALLM.Audio{
  metadata: map(),
  mime_type: String.t() | nil,
  source: source()
}

Functions

from_base64(encoded, mime_type)

@spec from_base64(String.t(), String.t()) :: t()

Build an %Audio{} from a base64-encoded string plus an explicit MIME type.

The string is not decoded here. Invalid base64 surfaces later, as {:error, :invalid_base64} from to_binary/1 or size/1.

Examples

iex> audio = ALLM.Audio.from_base64("aGk=", "audio/mpeg")
iex> audio.source
{:base64, "aGk="}

from_binary(bytes, mime_type)

@spec from_binary(binary(), String.t()) :: t()

Build an %Audio{} from raw bytes plus an explicit MIME type.

Examples

iex> audio = ALLM.Audio.from_binary(<<0, 255>>, "audio/wav")
iex> audio.source
{:binary, <<0, 255>>}
iex> audio.mime_type
"audio/wav"

from_file(path)

@spec from_file(Path.t()) :: t()

Build an %Audio{} from a local filesystem path.

Does no I/O: the path is stored as given, so a path that does not exist yet builds fine. :mime_type is inferred from the lowercased extension, or nil when the extension is missing or not in the table.

Examples

iex> audio = ALLM.Audio.from_file("/tmp/clip.MP3")
iex> audio.source
{:file, "/tmp/clip.MP3"}
iex> audio.mime_type
"audio/mpeg"

iex> ALLM.Audio.from_file("/tmp/clip.xyz").mime_type
nil

size(audio)

@spec size(t()) ::
  {:ok, non_neg_integer()}
  | {:error, :invalid_base64 | :invalid_source | File.posix()}

The audio payload's size in bytes, without reading a file.

  • {:binary, b} measures the bytes.
  • {:base64, s} decodes and measures, or returns {:error, :invalid_base64}.
  • {:file, path} stats the file and never reads it; a missing file returns {:error, :enoent}, and a directory returns {:error, :eisdir}, the same error to_binary/1 gives for it.
  • Any other :source shape returns {:error, :invalid_source}.

Examples

iex> ALLM.Audio.size(ALLM.Audio.from_binary(<<1, 2, 3>>, "audio/wav"))
{:ok, 3}

iex> ALLM.Audio.size(ALLM.Audio.from_base64("%%%", "audio/wav"))
{:error, :invalid_base64}

to_binary(audio)

@spec to_binary(t()) ::
  {:ok, binary()} | {:error, :invalid_base64 | :invalid_source | File.posix()}

Resolve the audio's :source to raw bytes.

  • {:binary, b} returns {:ok, b}.
  • {:base64, s} decodes, or returns {:error, :invalid_base64}.
  • {:file, path} reads the file, returning File.read/1's error on failure ({:error, :enoent} for a missing file).
  • Any other :source shape (only reachable by building the struct by hand) returns {:error, :invalid_source}.

Examples

iex> ALLM.Audio.to_binary(ALLM.Audio.from_base64("aGk=", "audio/wav"))
{:ok, "hi"}

iex> ALLM.Audio.to_binary(ALLM.Audio.from_file("/definitely/not/here.mp3"))
{:error, :enoent}