@spec max_audio_bytes() :: pos_integer()

Return the largest audio clip, in bytes, this adapter will upload.

ElevenLabs documents the limit as "less than 5.0GB". This value was not probed. The upload goes through Req's multipart step, which holds the whole clip in memory, so a clip near the cap needs that much free memory.

Examples

iex> ALLM.Providers.ElevenLabs.Transcription.max_audio_bytes()
4_999_999_999

@spec prepare_request(
  ALLM.TranscriptionRequest.t(),
  keyword()
) :: {:ok, Req.Request.t()} | {:error, ALLM.Error.TranscriptionAdapterError.t()}

Return an unfired Req.Request configured exactly as transcribe/2 would fire it.

The pre-flight gates run first, so this is defined only for a request that passes them. Under opts[:adapter_opts][:transcription_script] it returns a stub error instead of delegating.

Examples

iex> req = ALLM.TranscriptionRequest.new(audio: ALLM.Audio.from_binary("ID3", "audio/mpeg"))
iex> {:ok, http} = ALLM.Providers.ElevenLabs.Transcription.prepare_request(req, api_key: "sk_x")
iex> URI.to_string(http.url)
"https://api.elevenlabs.io/v1/speech-to-text"

@spec stream_sample_rates() :: [pos_integer()]

Return the PCM sample rates, in Hz, stream_transcribe/3 accepts: the pcm_<rate> values of ElevenLabs' realtime audio_format parameter.

Examples

iex> ALLM.Providers.ElevenLabs.Transcription.stream_sample_rates()
[8_000, 16_000, 22_050, 24_000, 44_100, 48_000]

@spec stream_transcribe(
  ALLM.TranscriptionStreamRequest.t(),
  Enumerable.t(),
  keyword()
) ::
  {:ok, Enumerable.t(ALLM.TranscriptionEvent.t())}
  | {:error, ALLM.Error.TranscriptionAdapterError.t()}

Transcribe PCM audio as it arrives, over ElevenLabs' realtime WebSocket wss://<host>/v1/speech-to-text/realtime.

input is an enumerable of PCM16 little-endian mono binaries at request.sample_rate, and :commit markers. Returns {:ok, enumerable} of ALLM.TranscriptionEvent values, or {:error, %ALLM.Error.TranscriptionAdapterError{}} from a pre-flight gate. The gates run in this order, then ALLM.Keys.fetch!/2, before the enumerable is returned:

1. Request shape. ALLM.Validate.transcription_stream_request/1; a failure is :invalid_request with the validator's errors on metadata.errors.
2. Sample rate. request.sample_rate must be in stream_sample_rates/0, else :invalid_request with metadata.sample_rate.
3. Language hold. adapter_opts[:language_hold_ms] must be a positive integer (milliseconds), else :invalid_request with metadata.field: :language_hold_ms. Checked whether or not the request opts in to the hold.

No socket is opened until the enumerable is reduced. The stream then connects, waits for the server's session_started (emitted as :transcription_started, session_id included), and only then starts reducing input in a helper process (see ALLM.Providers.Support.InputPump). A bad key is not refused at the upgrade: ElevenLabs answers 101 and then an auth_error frame, which ends the stream with :authentication_failed before the input is reduced.

Model resolution. model_id is request.model, else "scribe_v2_realtime", the only model the realtime endpoint accepts. The engine's transcription_model is never used here: the batch and realtime model names are disjoint.

Wire. The key goes in the upgrade request's xi-api-key header, never in the URL. The URL carries model_id, audio_format (pcm_<rate>), commit_strategy (vad or manual) and language_code (when request.language is set), plus every request.options entry as a further query parameter (for example %{"include_timestamps" => true}); an option named model_id, audio_format or commit_strategy is dropped. With timestamps: true or logprobs: true, include_timestamps=true is set by the adapter and overrides the option. Audio goes out as {"message_type": "input_audio_chunk", "audio_base_64": …, "commit": false, "sample_rate": rate}, one frame per input chunk and at most 1000 ms of audio per frame (a longer chunk is split). Chunk boundaries are the caller's: a chunk may end mid-sample, and the odd byte is carried into the next one, so only whole samples are sent. :commit sends an empty-audio frame with "commit": true.

End of input. When input is exhausted, audio sent since the last commit is committed, and the stream waits (up to :stream_timeout) for the transcript of every commit it sent before it emits :transcription_completed and closes. ElevenLabs refuses a commit that covers less than 0.3 s of audio with a commit_throttled frame (observed 2026-09-27); after the end of input that means there was nothing left to transcribe, and the stream completes normally. Mid-stream, a :commit the provider throttles ends the stream with :rate_limited.

Events. :partial_transcript for each server partial (a partial replaces the previous one), :committed_transcript for each server committed_transcript frame, in commit order. A committed_transcript_with_timestamps frame (sent only when include_timestamps is on, through a span flag or options, and carrying a language only with "include_language_detection" => true, observed 2026-09-27) never emits an event of its own: its language_code becomes the :language of the segment it belongs to, and with a span flag set its words become the segment's :spans. The two frames of a segment are paired by commit order (the n-th timestamped frame belongs to the n-th committed segment), never by comparing texts, so two segments with the same text each keep their own language. A warning frame is logged with Logger.warning/1 and emits nothing.

Spans. With timestamps: true or logprobs: true every :committed_transcript carries a :spans key, built as on the batch path, with word times relative to the start of the session's audio (the 2026-09-28 two-segment recording shows the second segment's words starting after the first's end). A segment's :spans is nil when its words cannot be trusted: its timestamped frame never arrived before the segment was released, that frame's trimmed text differs from the segment's (the commit-order pairing went wrong), or its words is malformed or missing on a non-blank text. A blank segment whose frame has no words has spans: []. :transcription_completed then carries :spans, the committed segments' non-nil spans concatenated in commit order ([] when there are none). With both flags off neither event has a :spans key, even when options turns include_timestamps on.

ElevenLabs sends the two frames of a segment in either order, most often the plain one first (3 of the 4 segments logged on 2026-09-27, the recorded rt_fox session among them), so:

• When a span flag is true, or options sets "include_timestamps" or "include_language_detection" (to true or "true"), each :committed_transcript is held until its timestamped frame arrives and is then emitted with that frame's language (and spans), so every segment may wait up to 1000 ms. The hold is bounded: the segment is released, with language: nil (and spans: nil), when the next segment's committed_transcript arrives, when the stream ends with an error, when :stream_timeout passes, or after 1000 ms (adapter_opts[:language_hold_ms]), whichever comes first. A session that is complete but for a held segment completes when the segment is released, even if :stream_timeout is the deadline that released it. A timestamped frame that arrives first is kept, and its segment is emitted at once. Holding never drops or reorders a segment, and :transcription_completed waits for a held segment.
• Otherwise (the default) nothing is held: each segment is emitted on its committed_transcript frame, latency first, with language only if its timestamped frame came first, which without those options it does not, so :language is nil.

A timestamped frame whose segment was already emitted is dropped. :transcription_completed carries the committed segments (each trimmed, empties dropped, joined with one space), the last language a segment carried, duration_seconds computed from the bytes sent (bytes / (sample_rate * 2)), opts[:request_id] and request.metadata; usage is all-nil (the protocol reports none).

Failures end the stream with one {:error, _}: :invalid_request for the input rules of ALLM.TranscriptionStreamAdapter (metadata.cause :invalid_input_chunk, including an input whose total length is odd, :input_raised or :input_crashed), a classified server error frame (by its message_type), :network_error for a transport failure or a close before the end, and :timeout.

Options: :stream_timeout (default 60000 ms) is the silence allowed between two messages, where a server frame and an input chunk both count. :connect_timeout bounds the upgrade. :ws_module is the ALLM.Providers.Support.WebSocket implementation (default ALLM.Providers.Support.WebSocket.Mint). adapter_opts[:input_window] is the input pump's credit window. adapter_opts[:language_hold_ms] (default 1000) bounds the language hold described under Events; it must be a positive integer (ms), and any other value is refused synchronously with :invalid_request.

Halting the stream closes the socket, stops the input pump and removes the socket's and the pump's pending messages from the calling process's mailbox. The realtime protocol has no keep-alive, so none is sent.

opts[:adapter_opts][:transcription_script] hands the call to ALLM.Providers.FakeTranscription.stream_transcribe/3 before any gate runs, with this adapter's stream_sample_rates/0 as adapter_opts[:stream_sample_rates].

Examples

iex> req = ALLM.TranscriptionStreamRequest.new(sample_rate: 24_000)
iex> opts = [adapter_opts: [transcription_script: [{:ok, "the quick fox"}]]]
iex> {:ok, events} = ALLM.Providers.ElevenLabs.Transcription.stream_transcribe(req, [<<0, 0>>], opts)
iex> for {:committed_transcript, %{text: t}} <- events, do: t
["the quick fox"]

iex> req = ALLM.TranscriptionStreamRequest.new(sample_rate: 24_000, timestamps: true)
iex> opts = [adapter_opts: [transcription_script: [{:ok, "hi"}]]]
iex> {:ok, events} = ALLM.Providers.ElevenLabs.Transcription.stream_transcribe(req, [<<0, 0>>], opts)
iex> {:transcription_completed, completed} = List.last(Enum.to_list(events))
iex> for s <- completed.spans, do: {s.text, s.end_seconds}
[{"hi", 0.5}]

iex> req = ALLM.TranscriptionStreamRequest.new(sample_rate: 11_025)
iex> {:error, err} = ALLM.Providers.ElevenLabs.Transcription.stream_transcribe(req, [], [])
iex> {err.reason, err.metadata.sample_rate}
{:invalid_request, 11_025}

@spec transcribe(
  ALLM.TranscriptionRequest.t(),
  keyword()
) ::
  {:ok, ALLM.TranscriptionResponse.t()}
  | {:error, ALLM.Error.TranscriptionAdapterError.t()}

Transcribe request.audio against ElevenLabs.

Returns {:ok, %ALLM.TranscriptionResponse{}} or {:error, %ALLM.Error.TranscriptionAdapterError{}}. The one exception is ALLM.Keys.fetch!/2, which raises %ALLM.Error.EngineError{reason: :missing_key} by design; every pre-flight gate runs ahead of it.

Injected defaults: model_id is "scribe_v2" when the request leaves :model nil, and the receive timeout is 120 s when opts[:request_timeout] is absent.

Refused before any I/O with :unsupported_feature: a non-nil :prompt (ElevenLabs has no prompt field).

Spans: timestamps: true and logprobs: true are both honoured; see "Word timings and log-probabilities" in the module doc. Under the transcription_script hand-off the Fake is told both flags are supported.

No retries: one HTTP attempt per call, whatever the error.

Examples

iex> audio = ALLM.Audio.from_binary("ID3", "audio/mpeg")
iex> req = ALLM.TranscriptionRequest.new(audio: audio)
iex> opts = [adapter_opts: [transcription_script: [{:ok, "hello"}]]]
iex> {:ok, resp} = ALLM.Providers.ElevenLabs.Transcription.transcribe(req, opts)
iex> resp.text
"hello"

iex> audio = ALLM.Audio.from_binary("ID3", "audio/mpeg")
iex> req = ALLM.TranscriptionRequest.new(audio: audio, logprobs: true)
iex> opts = [adapter_opts: [transcription_script: [{:ok, "hello"}]]]
iex> {:ok, resp} = ALLM.Providers.ElevenLabs.Transcription.transcribe(req, opts)
iex> for s <- resp.spans, do: {s.text, s.start_seconds, s.logprob}
[{"hello", nil, -0.1}]

iex> audio = ALLM.Audio.from_binary("ID3", "audio/mpeg")
iex> req = ALLM.TranscriptionRequest.new(audio: audio, prompt: "names: Ada")
iex> {:error, err} = ALLM.Providers.ElevenLabs.Transcription.transcribe(req, [])
iex> {err.reason, err.metadata.field}
{:unsupported_feature, :prompt}