ALLM.Providers.OpenAI.Transcription (allm v0.6.1)

Copy Markdown View Source

OpenAI speech-to-text adapter. Implements ALLM.TranscriptionAdapter against POST /v1/audio/transcriptions.

Layer B — runtime. Wire it with ALLM.Engine.new(transcription_adapter: ALLM.Providers.OpenAI.Transcription) and call it through ALLM.transcribe/3. The key resolves via ALLM.Keys.fetch!(:openai, opts) after the pre-flight gates, so no key ever lives on the engine.

req = ALLM.TranscriptionRequest.new(audio: ALLM.Audio.from_file("clip.mp3"))
{:ok, resp} = ALLM.Providers.OpenAI.Transcription.transcribe(req, api_key: "sk-...")
resp.text

Wire-field map

Observed live on 2026-09-24 by scripts/record_openai_audio_fixtures.exs, whose probe halts the recording pass on any mismatch.

ConcernOpenAI
EndpointPOST https://api.openai.com/v1/audio/transcriptions, multipart/form-data
Authauthorization: Bearer <key>
filethe audio bytes. Named by the file's basename for a {:file, path} source, else audio.<ext> from the MIME type
model:model, or gpt-transcribe when nil
response_formatalways json
language, promptsent when set
Optionseach ALLM.TranscriptionRequest.options entry becomes one form field (a list becomes one field per element, each under the bare key; OpenAI names array parameters with a [] suffix, so pass "timestamp_granularities[]" as the key where the endpoint expects it); never overrides the fields above; response_format is dropped
include[]logprobs, sent when logprobs: true; never overridden by an option of the same name
Response{"text", "usage", "languages"?, "logprobs"?}
Log-probabilitieslogprobs[] = {token, logprob, bytes}, present only when include[]=logprobs was sent (confirmed on gpt-transcribe and gpt-4o-mini-transcribe, 2026-09-28)
Usage{"type": "duration", "seconds"} (whisper-1, gpt-transcribe) → :duration_seconds; {"type": "tokens", …} (gpt-4o-mini-transcribe) → :usage
Languagelanguages[0].code → :language (gpt-transcribe only)
Correlationx-request-id response header
Size limit25 MiB on the whole multipart body; see max_audio_bytes/0
Durationno duration cap was found: an 1800-second clip was accepted by gpt-transcribe
Unknown fieldsignored (200), so a mistyped :options key does nothing and raises no error

Accepted formats, per OpenAI's own error text: flac, m4a, mp3, mp4, mpeg, mpga, oga, ogg, wav, webm. Other formats are forwarded and answered with a 400, surfaced as :invalid_request.

Adapter-injected defaults

  • model defaults to "gpt-transcribe" when :model is nil.
  • The HTTP receive timeout defaults to 120 s when opts[:request_timeout] is absent.

Pre-flight gates

In this order, before any HTTP I/O and before ALLM.Keys.fetch!/2:

  1. Resolvable. Audio whose bytes cannot be resolved (a missing file, a directory, invalid base64, an off-shape source, or an :audio that is not an %ALLM.Audio{}) → :invalid_request with metadata.cause and metadata.field: :audio.
  2. Size. More than max_audio_bytes/0 bytes → :invalid_request with metadata.count and metadata.max. A {:file, path} source is measured with a stat, never read.
  3. Filename. A non-file source whose :mime_type is nil or not a known audio type → :invalid_request with metadata.mime_type. Parameters and case are ignored, so "audio/webm;codecs=opus" is sent as audio.webm. OpenAI picks the decoder from the upload's filename extension, so such a clip would be sent under a name it rejects.
  4. Span flags. timestamps: true → :unsupported_feature with metadata.field: :timestamps. logprobs: true passes.

Token log-probabilities

With logprobs: true the request carries include[]=logprobs and response.spans is one %ALLM.TranscriptSpan{kind: :token} per entry of the response's logprobs list, in order: token → :text, logprob → :logprob, and nil times. With both flags off spans is nil and any logprobs in the body is left on :raw only.

Which models return log-probabilities is OpenAI's decision; this adapter keeps no model list. A 200 whose body has no logprobs list is:

  • {:ok, response} with spans: [] when the transcript is blank (silence: observed 2026-09-28 on gpt-transcribe, which omits the key and returns text: "");
  • otherwise :unsupported_feature with metadata.cause: :absent_from_response, metadata.field: :logprobs and the transcript on metadata.text, so the paid transcript is not lost. A model that does not support log-probabilities (for example whisper-1) lands here.

A logprobs value that is not a list of objects with a string token is :malformed_response. Flags are compared with == true, so logprobs: "yes" is off.

Response

:text is the transcript ("" for silence). :raw is the provider body. :model is the model that was sent. opts[:request_id] is reflected onto response.request_id; when absent, OpenAI's x-request-id header is used. request.metadata round-trips onto response.metadata.

The x-request-id fallback is unreachable through ALLM.transcribe/3

The façade always supplies opts[:request_id] (it generates one when the caller does not), so on that path response.request_id is the façade's id and OpenAI's own correlation id is never observed. It surfaces only on a direct transcribe/2 / decode_response/4 call that omits opts[:request_id]. Identical to ALLM.Providers.OpenAI.Moderation.

No retries

This adapter makes one HTTP attempt per call and returns every classified error, retryable or not, as {:error, _}. Each attempt re-uploads the whole clip, so the only retry loop is ALLM.transcribe/3's (3 attempts at the default policy). A direct transcribe/2 caller who wants retries wraps the call.

Error-struct hygiene

No raw response body, request header or Authorization value is copied into %ALLM.Error.TranscriptionAdapterError{}, and there is no body preview. The provider's message, code and type pass a redactor that replaces key-shaped tokens with [REDACTED]. OpenAI's real 401 is sent as text/plain and masks the key it echoes (observed 2026-09-24).

Not every error is JSON-encodable

The struct derives Jason.Encoder, but a :timeout, :network_error or invalid-JSON :malformed_response error carries an exception struct (Req.TransportError, Jason.DecodeError) on :cause, and Jason.encode!/1 raises Protocol.UndefinedError on it. Drop or Exception.message/1 the :cause before encoding. The image, embedding and moderation adapter errors share this limitation.

Test-injection escape hatch

transcribe/2 honours opts[:adapter_opts][:transcription_script]: when the key is present, the call is handed to ALLM.Providers.FakeTranscription.transcribe/2 BEFORE any of this adapter's gates run, with adapter_opts[:max_audio_bytes] set to this adapter's own max_audio_bytes/0 so a real clip is not rejected by the Fake's small default cap, and adapter_opts[:span_flags] set to [:logprobs] so the Fake refuses timestamps: true exactly as this adapter does. prepare_request/2 returns a stub error under the same key. Refusals match this adapter, but span kinds do not: a scripted logprobs: true call returns the Fake's :word spans, while this adapter returns :token spans. ALLM.TranscriptionResponse.mean_logprob/1 treats both alike.

Summary

Functions

max_audio_bytes()

Return the largest audio clip, in bytes, this adapter will upload.

prepare_request(request, opts)

Return an unfired Req.Request configured exactly as transcribe/2 would fire it.

transcribe(request, opts)

Transcribe request.audio against OpenAI.

Functions

max_audio_bytes()

@spec max_audio_bytes() :: pos_integer()

Return the largest audio clip, in bytes, this adapter will upload.

OpenAI caps the whole multipart request body at 25 MiB (26,214,400 bytes). This cap is 25 MiB minus 64 KiB, the largest file part a live probe on 2026-09-24 saw accepted, leaving room for the other form fields. A very long :prompt can still push a clip just under the cap over the provider's limit; that surfaces as :invalid_request from a 413.

Examples

iex> ALLM.Providers.OpenAI.Transcription.max_audio_bytes()
26_148_864

prepare_request(request, opts)

@spec prepare_request(
  ALLM.TranscriptionRequest.t(),
  keyword()
) :: {:ok, Req.Request.t()} | {:error, ALLM.Error.TranscriptionAdapterError.t()}

Return an unfired Req.Request configured exactly as transcribe/2 would fire it.

The pre-flight gates run first, so this is defined only for audio that passes them. Under opts[:adapter_opts][:transcription_script] it returns a stub error instead of delegating.

Examples

iex> req = ALLM.TranscriptionRequest.new(audio: ALLM.Audio.from_binary("ID3", "audio/mpeg"))
iex> {:ok, http} = ALLM.Providers.OpenAI.Transcription.prepare_request(req, api_key: "sk-x")
iex> URI.to_string(http.url)
"https://api.openai.com/v1/audio/transcriptions"

transcribe(request, opts)

@spec transcribe(
  ALLM.TranscriptionRequest.t(),
  keyword()
) ::
  {:ok, ALLM.TranscriptionResponse.t()}
  | {:error, ALLM.Error.TranscriptionAdapterError.t()}

Transcribe request.audio against OpenAI.

Returns {:ok, %ALLM.TranscriptionResponse{}} or {:error, %ALLM.Error.TranscriptionAdapterError{}}. The one exception is ALLM.Keys.fetch!/2, which raises %ALLM.Error.EngineError{reason: :missing_key} by design; every pre-flight gate runs ahead of it.

Injected defaults: model is "gpt-transcribe" when the request leaves it nil, and the receive timeout is 120 s when opts[:request_timeout] is absent.

No retries: one HTTP attempt per call, whatever the error. Wrap the call yourself if you want retries, or go through ALLM.transcribe/3, which retries up to 3 times at the default policy.

No duration cap was found: a live probe on 2026-09-24 sent an 1800-second clip to gpt-transcribe and got a 200. Only the byte cap in max_audio_bytes/0 is enforced here.

Examples

iex> audio = ALLM.Audio.from_binary("ID3", "audio/mpeg")
iex> req = ALLM.TranscriptionRequest.new(audio: audio)
iex> opts = [adapter_opts: [transcription_script: [{:ok, "hello"}]]]
iex> {:ok, resp} = ALLM.Providers.OpenAI.Transcription.transcribe(req, opts)
iex> resp.text
"hello"

iex> req = ALLM.TranscriptionRequest.new(audio: ALLM.Audio.from_file("/nonexistent.mp3"))
iex> {:error, err} = ALLM.Providers.OpenAI.Transcription.transcribe(req, [])
iex> {err.reason, err.metadata.cause}
{:invalid_request, :enoent}

Word timings are refused before any key is needed; token log-probabilities (logprobs: true) are supported (see the module documentation):

iex> audio = ALLM.Audio.from_binary("ID3", "audio/mpeg")
iex> req = ALLM.TranscriptionRequest.new(audio: audio, timestamps: true)
iex> {:error, err} = ALLM.Providers.OpenAI.Transcription.transcribe(req, [])
iex> {err.reason, err.metadata.field}
{:unsupported_feature, :timestamps}