@spec assistant(String.t()) :: ALLM.Message.t()

Build an assistant-role %ALLM.Message{} from a text string.

Examples

iex> ALLM.assistant("hello")
%ALLM.Message{role: :assistant, content: "hello", name: nil, tool_call_id: nil, metadata: %{}}

@spec chat(ALLM.Engine.t(), ALLM.Thread.t() | [ALLM.Message.t()], keyword()) ::
  {:ok, ALLM.ChatResult.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.AdapterError.t()
     | ALLM.Error.ValidationError.t()}

Run a multi-turn chat loop against the engine and return a %ALLM.ChatResult{}.

thread_or_messages is either an %ALLM.Thread{} or a list of %ALLM.Message{} (normalised via ALLM.Thread.from_messages/1). The thread is validated via ALLM.Validate.thread/1 at entry.

Mode

• :auto (default) — the loop executes tool calls automatically. Each step appends tool-result messages to the thread before the next adapter call. Halt reasons follow the table below.
• :manual — the FIRST step whose response carries finish_reason: :tool_calls halts with halted_reason: :manual_tool_calls. The caller submits tool results via a fresh chat/3 call with the augmented thread (no executor runs). Pure-text steps under :manual continue normally.

:max_turns precedence

The loop bound resolves at entry through this chain (call opts wins on the left):

call opts > engine.params[:max_turns] > Application.get_env(:allm, :max_turns) > library default 8

max_turns must be a pos_integer; non-positive integers raise ArgumentError.

Sampling params (:max_tokens, :temperature, …)

Sampling params resolve through the same precedence — call opts win over engine.params — and are folded onto the built %Request{}: :max_tokens and :temperature land on their typed fields; every other opaque param (:top_p, :reasoning_effort, …) rides on request.options for the adapter body-builder to merge. An unset :max_tokens stays nil, so the provider's default applies as a floor (Anthropic injects 1024) — raise it via Engine.new(params: %{max_tokens: N}) or a per-call max_tokens: opt when a turn needs more room, or it truncates (finish_reason: :length) before tools run.

Prompt caching (:prompt_cache)

prompt_cache: asks the provider to cache the stable prefix of the conversation. It resolves like the sampling params (call opts win over engine.params) and lands on the typed request.prompt_cache field, never on request.options. The value is normalized to %{key: key, retention: :short | :long}:

The key defaults to :session_id only when it is a non-empty binary (otherwise the key stays nil), and only when caching was asked for: a session_id: alone never turns caching on. An explicit key: "" is still rejected. On engine.params, prefer the map form: an engine carrying a keyword-list prompt_cache cannot be encoded to JSON, while a keyword list as a call opt is fine. ALLM.Session passes session.id as :session_id, so a session's id becomes its cache key. Each provider adapter's generate/2 docs say what it sends for the field (ALLM.Providers.OpenAI, ALLM.Providers.Anthropic, ALLM.Providers.Gemini).

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> params: %{prompt_cache: %{retention: :long}},
...> adapter_opts: [script: [{:text, "ok"}, {:finish, :stop}], record: self()]
...>)
iex> {:ok, _} = ALLM.chat(engine, [ALLM.user("hi")], session_id: "recipe-42")
iex> receive do
...>   {:allm_fake_record, request, _opts} -> request.prompt_cache
...> end
%{key: "recipe-42", retention: :long}

:halt_when semantics

:halt_when is a (StepResult.t -> boolean) callback invoked AFTER the step's thread mutation has been applied. It is the LAST per-step gate consulted — ask-user, handler {:halt, _, _}, on_tool_error: :halt, :manual_tool_calls, and adapter finish_reason ∈ {:stop, :error, :length, :content_filter} all preempt it. Exceptions raised inside halt_when propagate to the caller of chat/3; they are NOT caught.

:on_tool_error

Atom forms :continue (default) and :halt are the common cases. The function form (ToolCall.t, term -> {:continue, term} | :halt) is invoked synchronously inside the per-tool task after the handler's return / encoder failure resolves to an error term: {:continue, replacement} encodes replacement as the tool-result content; :halt halts the batch with halted_reason: :tool_error. An invalid return shape or a raise from inside the function is wrapped as %ALLM.Error.ToolError{reason: :invalid_return} and treated as :halt.

:on_event scope

:on_event observes only adapter-emitted events (text deltas, tool-call deltas, message bookends, :raw_chunk, adapter-emitted :error). Chat-layer events (:tool_execution_*, :tool_result_encoded, :ask_user_requested, :tool_halt, :step_completed, :chat_completed) are NOT delivered to :on_event.

Halt-reason table

Mixed-bucket re-issue (per-tool manual)

When mode: :auto and at least one called tool has manual: true, the loop halts with halted_reason: :manual_tool_calls after running the auto-bucket tools. The returned result.thread carries the assistant message AND the auto-bucket :tool messages — but NOT placeholder messages for the manual ids. Naively re-issuing chat/3 on result.thread sends a malformed request to the provider (assistant tool_calls with no matching :tool messages for the manual ids), surfacing as %ALLM.Error.AdapterError{reason: :invalid_request}.

Callers MUST append a :tool message for each id in result.metadata.manual_tool_calls before re-issuing:

{:ok, result} = ALLM.chat(engine, [ALLM.user("...")])

# result.halted_reason == :manual_tool_calls
# result.metadata.manual_tool_calls == [%ToolCall{id: "cm", ...}]

# Resolve each manual call out-of-band, then append a :tool message.
tool_msg = %ALLM.Message{
  role: :tool,
  content: "approved",
  tool_call_id: "cm"
}
augmented = ALLM.Thread.add_message(result.thread, tool_msg)

{:ok, final} = ALLM.chat(engine, augmented)

The ALLM.Session API (ALLM.Session.start/3 + ALLM.Session.submit_tool_result/3) enforces this discipline automatically; raw chat/3 callers must guard by hand. Whole-loop mode: :manual callers are unaffected — every tool call surfaces on result.final_response.tool_calls, no auto bucket exists.

Examples

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> adapter_opts: [
...> scripts: [
...> [{:tool_call, id: "c0", name: "echo", arguments: %{"x" => 1}},
...> {:finish, :tool_calls}],
...> [{:text, "done"}, {:finish, :stop}]
...> ]
...> ],
...> tools: [ALLM.tool(
...> name: "echo",
...> description: "",
...> schema: %{},
...> handler: fn args -> {:ok, args} end
...>)]
...>)
iex> {:ok, %ALLM.ChatResult{} = result} = ALLM.chat(engine, [ALLM.user("echo please")])
iex> {result.halted_reason, length(result.steps)}
{:completed, 2}

@spec classification_request(
  ALLM.ClassificationRequest.state(),
  keyword()
) :: ALLM.ClassificationRequest.t()

Build an %ALLM.ClassificationRequest{} from the state to classify.

The state is a string, a JSON object (map) or a JSON array (list) — the text the questions are asked about.

Options

Only ALLM.ClassificationRequest field names are read: :questions, :model, :options, :metadata. Every other key is ignored, which is what lets classify/3 forward its own call-control opts (:request_id, :request_timeout, :retry, :adapter_opts, :api_key, :stream) through this function without them landing on the struct.

Atom question ids in :questions are converted to strings, because an atom key does not survive a JSON round trip. Nothing else is done to the questions.

No validation runs here — call ALLM.Validate.classification_request/1 if you want the field rules checked before dispatch. classify/3 calls it for you.

Examples

iex> q = ALLM.ClassificationQuestion.yes_no("Is a refund requested?")
iex> req = ALLM.classification_request("Please refund me.", questions: %{refund: q}, model: "jev-1.13.0")
iex> {req.state, Map.keys(req.questions), req.model}
{"Please refund me.", ["refund"], "jev-1.13.0"}

@spec classify(
  ALLM.Engine.t(),
  ALLM.ClassificationRequest.state() | ALLM.ClassificationRequest.t(),
  keyword()
) ::
  {:ok, ALLM.ClassificationResponse.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.ClassificationAdapterError.t()}

Ask typed questions about a piece of state using the engine's :classification_adapter.

Layer-C façade. Two input shapes:

• State — a string, a map (JSON object) or a list (JSON array). Opts named in classification_request/2 (:questions, :model, :options, :metadata) lift onto the built request; everything else is a call-control opt.
• Pre-built %ALLM.ClassificationRequest{} — dispatched verbatim; opts are NOT merged onto it.

Each question is an ALLM.ClassificationQuestion — pick one option (choice/2), score on ordered levels (score/2), or yes/no (yes_no/2). The response carries one typed ALLM.ClassificationAnswer per question id; read one with ALLM.ClassificationResponse.answer/2.

The library does not decide thresholds. It returns the probabilities, score position and confidence the provider reports; what counts as "confident enough" lives in your code.

One call, many questions

Every question in the request is answered in the same call, against the same state. There is no chunking and no question-count cap in the façade: splitting questions across calls would re-send the whole state each time. Put every question you have about a state into one request.

Model resolution

The model comes from the classification slot, never from the chat model: request.model, else engine.classification_model, else the adapter's own documented default. engine.model is not consulted — a chat model name is not a classification model name. On the state shape opts[:model] sets request.model; a pre-built request's :model is authoritative.

Gate order

1. {:error, %ALLM.Error.EngineError{reason: :no_classification_adapter}} when engine.classification_adapter == nil — first, so a misconfigured engine never surfaces as a request problem.
2. Validation (below).
3. Dispatch to ALLM.ClassificationAdapter.classify/2, under the retry policy below.

Validation

ALLM.Validate.classification_request/1 runs before dispatch, returning {:error, %ALLM.Error.ValidationError{reason: :invalid_classification_request}} on failure: empty state, no questions, a malformed question, or a state or question field that cannot be encoded as JSON. A keyword list or a struct passed as the state is rejected the same way rather than raising or being sent to the provider. Any other state (an integer, nil) raises FunctionClauseError.

Unknown opts — forwarded to the adapter

Any opt that is neither a request field nor a call-control opt is passed through to ALLM.ClassificationAdapter.classify/2 untouched, so provider-specific knobs need no façade change. :api_key is forwarded too; the adapter resolves the key itself.

Retry

:rate_limited, :provider_unavailable, :timeout and :network_error are retried under the engine's :retry policy; every other ALLM.Error.ClassificationAdapterError reason surfaces immediately. opts[:retry] is also forwarded to the adapter; an adapter that runs its own retry loop inside this one multiplies the two budgets. Read the adapter's own docs for whether it does.

Telemetry

The call runs inside an [:allm, :classify, …] span (see ALLM.Telemetry). :start and :stop metadata carry question_count; :stop adds the answer_count measurement (0 on error) and usage (nil on error).

Raises

• %ALLM.Error.EngineError{reason: :missing_key} when the adapter cannot resolve an API key. It is raised, not returned, as on every other façade.
• ArgumentError, naming the adapter, when ALLM.ClassificationAdapter.classify/2 returns anything other than {:ok, %ALLM.ClassificationResponse{}} or {:error, %ALLM.Error.ClassificationAdapterError{}} (invariant 1). The error union above describes conforming adapters only.

request_id precedence

opts[:request_id] wins over an auto-generated id. The id is forwarded to the adapter and filled onto response.request_id IFF the adapter left it nil; an adapter-populated id is preserved.

:stream opt is silently dropped

Classification is request/response — there is no streaming variant. Passing stream: true does not error; the opt is ignored.

Examples

iex> q = ALLM.ClassificationQuestion.choice("Which team?", ["billing", "technical"])
iex> engine = ALLM.Engine.new(
...> classification_adapter: ALLM.Providers.FakeClassification,
...> adapter_opts: [classification_script: [{:answers, %{"team" => "technical"}}]]
...>)
iex> {:ok, resp} = ALLM.classify(engine, "The app crashes on login.", questions: %{team: q})
iex> ALLM.ClassificationResponse.answer(resp, :team).choice
"technical"

iex> q = ALLM.ClassificationQuestion.yes_no("Is a refund requested?")
iex> {:error, %ALLM.Error.EngineError{reason: :no_classification_adapter}} =
...> ALLM.classify(ALLM.Engine.new(), "Please refund me.", questions: %{refund: q})
iex> :ok
:ok

@spec edit_image(
  ALLM.Engine.t(),
  ALLM.Image.t() | [ALLM.Image.t()],
  String.t(),
  keyword()
) ::
  {:ok, ALLM.ImageResponse.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.ImageAdapterError.t()}

Edit a base image (optionally with a mask) against the engine's :image_adapter.

Three call shapes:

• edit_image(engine, base_image, prompt) — single base, no mask; builds %ImageRequest{operation: :edit, input_images: [base], mask: nil}.
• edit_image(engine, [base, mask], prompt) — 2-element list; both images become :input_images, :mask stays nil. The list form does NOT auto-promote the second element to :mask — use the explicit mask: keyword for that.
• edit_image(engine, base, prompt, mask: mask) — explicit mask keyword; builds input_images: [base], mask: mask.

Returns {:error, %EngineError{reason: :no_image_adapter}} when the engine has no image adapter (first gate, before any other validation).

Forwards opts (n, size, quality, etc.) onto the request struct via ALLM.ImageRequest.new/1. See generate_image/3 for the full request_id and :stream-drop semantics — they apply identically.

Examples

iex> img = ALLM.Image.from_binary(<<137, 80, 78, 71>>, "image/png")
iex> engine = ALLM.Engine.new(
...> image_adapter: ALLM.Providers.FakeImages,
...> adapter_opts: [image_script: [{:ok, [img]}]]
...>)
iex> base = ALLM.Image.from_binary(<<1, 2, 3>>, "image/png")
iex> {:ok, %ALLM.ImageResponse{images: [_]}} =
...> ALLM.edit_image(engine, base, "make sky pink")
iex> :ok
:ok

@spec embed(
  ALLM.Engine.t(),
  String.t() | [String.t()] | ALLM.EmbeddingRequest.t(),
  keyword()
) ::
  {:ok, ALLM.EmbeddingResponse.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.EmbeddingAdapterError.t()}

Turn text into vectors using the engine's :embed_adapter.

Layer-C façade. Three input shapes:

• Binary — sugar for a one-element batch. Opts named in embedding_request/2 (:model, :dimensions, :task_type, :truncate, :options, :metadata) lift onto the built request; everything else is a call-control opt.
• List of binaries — a batch, chunked transparently (see below). Same opt-lifting rule.
• Pre-built %ALLM.EmbeddingRequest{} — dispatched verbatim; opts are NOT merged onto it.

Unknown opts — forwarded to the adapter

Any opt that is neither a request field nor a call-control opt is passed through to ALLM.EmbeddingAdapter.embed/2 untouched, so provider-specific knobs need no façade change.

On success the response's embeddings are in input order: Enum.at(ALLM.EmbeddingResponse.vectors(response), i) is the vector for Enum.at(input, i), whatever order the provider returned items in and however many HTTP requests the call became.

Adapter-presence gate

Returns {:error, %ALLM.Error.EngineError{reason: :no_embed_adapter}} when engine.embed_adapter == nil. This is the first gate — it fires ahead of validation and capability pre-flight, so a misconfigured engine never surfaces as a request problem.

Validation policy

Unlike generate_image/3, this façade DOES call ALLM.Validate.embedding_request/1, returning {:error, %ALLM.Error.ValidationError{reason: :invalid_embedding_request}} on failure. An empty-string input is a guaranteed provider rejection, and in a chunked call it should fail before 49 successful round-trips are spent on the other chunks.

Batching

Adapters declare a per-request cap (ALLM.EmbeddingAdapter.max_batch_size/0) — 2048, 100, and 1000 across the bundled providers. Longer input lists are split into that many items per request, dispatched sequentially, and merged back into a single response. response.metadata.chunk_count records how many requests the call became; the [:allm, :embed, :stop] telemetry measurement of the same name reports it too. :chunk_count is reserved on response.metadata — a request-supplied metadata key of that name is overwritten.

Sequential dispatch is deliberate: firing 50 chunks concurrently is the fastest route to a rate-limit storm, and it also bounds the call to one in-flight request at a time.

Merging keeps the first chunk's :raw only and sums every numeric %ALLM.Usage{} counter across chunks (token counts and costs alike), merging :tool_usage / :extra as maps with the earlier chunk winning. No usage field is dropped, so usage does not depend on how many chunks the call became. A single-chunk call skips the merge entirely, so :raw survives intact for the common case.

Retry and time budgets are PER CHUNK

There is no aggregate ceiling on a chunked call. For a 5,000-input batch against a 100-item cap (50 chunks):

If you need a real bound, chunk the input yourself against the adapter's max_batch_size/0 and wrap each embed/3 call — the same loop that gives you resumability after a mid-batch failure.

Mid-batch failure

A failing chunk fails the whole call: no partial vectors are returned. The error is that chunk's own, with metadata.completed_chunks and metadata.completed_inputs added for diagnostics.

Non-conforming adapters raise

ALLM.EmbeddingAdapter.embed/2 must return {:ok, %ALLM.EmbeddingResponse{}} or {:error, %ALLM.Error.EmbeddingAdapterError{}} (invariant 2). Anything else raises ArgumentError naming the offending adapter rather than being laundered into this function's error union — the union above describes conforming adapters only.

request_id precedence

opts[:request_id] wins over an auto-generated id. The id is forwarded to the adapter and filled onto response.request_id IFF the adapter left it nil; an adapter-populated id is preserved.

:stream opt is silently dropped

Embeddings are request/response — there is no stream_embed/3, by design. Passing stream: true does not error; the opt is ignored.

Examples

iex> e = ALLM.Embedding.new(vector: [0.1, 0.2])
iex> engine = ALLM.Engine.new(
...> embed_adapter: ALLM.Providers.FakeEmbeddings,
...> adapter_opts: [embedding_script: [{:ok, [e]}]]
...>)
iex> {:ok, resp} = ALLM.embed(engine, "a kestrel")
iex> ALLM.EmbeddingResponse.vectors(resp)
[[0.1, 0.2]]

iex> engine = ALLM.Engine.new()
iex> {:error, %ALLM.Error.EngineError{reason: :no_embed_adapter}} =
...> ALLM.embed(engine, "a kestrel")
iex> :ok
:ok

@spec embedding_request(
  String.t() | [String.t()],
  keyword()
) :: ALLM.EmbeddingRequest.t()

Build an %ALLM.EmbeddingRequest{} from a string or a list of strings.

A bare string is normalised to a one-element list, so :input on the struct is always [String.t()] and no adapter or validator has to handle a union.

Options

Only ALLM.EmbeddingRequest field names are read: :model, :dimensions, :task_type, :truncate, :options, :metadata. Every other key is ignored, which is what lets embed/3 forward its own call-control opts (:request_id, :request_timeout, :retry, :adapter_opts, :api_key, :telemetry_metadata, :stream) through this function without them landing on the struct.

No validation runs here — call ALLM.Validate.embedding_request/1 if you want the field rules checked before dispatch. embed/3 calls it for you.

Examples

iex> req = ALLM.embedding_request("a kestrel", task_type: :search_document)
iex> req.input
["a kestrel"]
iex> req.task_type
:search_document

iex> req = ALLM.embedding_request(["one chunk", "another"], dimensions: 512)
iex> length(req.input)
2

@spec generate(ALLM.Engine.t(), ALLM.Request.t(), keyword()) ::
  {:ok, ALLM.Response.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.AdapterError.t()
     | ALLM.Error.ValidationError.t()}

Execute a non-streaming generation against the engine's adapter.

Implemented as a reducer over stream_generate/3 — the streaming path is the primitive. A mid-stream adapter error folds into response.finish_reason == :error with the error struct under response.metadata.error; pre-flight errors surface directly as {:error, struct} at the call site. Callers matching only {:error, _} will silently swallow rate limits, content-filter blocks, and stream cancellations — match on response.finish_reason == :error to handle mid-stream failures.

Options

Accepts the same options as stream_generate/3. :include_raw_chunks defaults to false but {:usage, _} raw chunks always survive the filter so response.usage is populated regardless. As there, prompt caching is set on the request (ALLM.Request's :prompt_cache field), not through a prompt_cache: opt.

Examples

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> adapter_opts: [script: [{:text, "hi"}, {:finish, :stop}]]
...>)
iex> req = ALLM.request([ALLM.user("say hi")])
iex> {:ok, response} = ALLM.generate(engine, req)
iex> {response.output_text, response.finish_reason}
{"hi", :stop}

@spec generate_image(ALLM.Engine.t(), String.t() | ALLM.ImageRequest.t(), keyword()) ::
  {:ok, ALLM.ImageResponse.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.ImageAdapterError.t()}

Generate one or more images against the engine's :image_adapter.

Layer-C façade. Two input shapes:

• Binary prompt — sugar over ALLM.image_request/2. Opts merge into the built %ALLM.ImageRequest{operation: :generate}.
• Pre-built %ALLM.ImageRequest{} — dispatched verbatim.

Adapter-presence gate

Returns {:error, %ALLM.Error.EngineError{reason: :no_image_adapter}} when engine.image_adapter == nil. This is the first gate; no other validation runs.

Validation policy

The façade does NOT call ALLM.Validate.image_request/1. Caller-opt-in only — mirrors request/2's no-validate precedent. A manually-built request that the validator would reject (e.g., empty prompt for :generate) still dispatches.

request_id precedence

opts[:request_id] wins over an auto-generated id from ALLM.Telemetry.request_id/0. The id is forwarded to the adapter via opts[:request_id]. After the call, response.request_id is filled from the forwarded id IFF the adapter left it nil; an adapter-populated :request_id (e.g. provider's x-request-id header) is preserved.

:stream opt is silently dropped

Image generation is non-streaming. Passing stream: true does not error — the opt is ignored.

Unknown opts

Forwarded to the adapter via opts (matches the chat-side Engine.resolve_params/2 pass-through pattern).

Examples

iex> img = ALLM.Image.from_binary(<<137, 80, 78, 71>>, "image/png")
iex> engine = ALLM.Engine.new(
...> image_adapter: ALLM.Providers.FakeImages,
...> adapter_opts: [image_script: [{:ok, [img]}]]
...>)
iex> {:ok, %ALLM.ImageResponse{images: [_]}} = ALLM.generate_image(engine, "a kestrel")
iex> :ok
:ok

iex> engine = ALLM.Engine.new()
iex> {:error, %ALLM.Error.EngineError{reason: :no_image_adapter}} =
...> ALLM.generate_image(engine, "a kestrel")
iex> :ok
:ok

@spec image_request(
  String.t(),
  keyword()
) :: ALLM.ImageRequest.t()

Build an %ALLM.ImageRequest{} from a prompt and keyword opts. Delegates to ALLM.ImageRequest.new/1 after putting :prompt last in the opts list — the positional prompt is authoritative.

Does not validate — call ALLM.Validate.image_request/1 to check operation-arity and field rules. Mirrors request/2's no-validate precedent: construction is composable, validation is an explicit step. Unknown opts raise KeyError via struct!/2.

Examples

iex> req = ALLM.image_request("a kestrel")
iex> {req.operation, req.prompt, req.n, req.response_format}
{:generate, "a kestrel", 1, :binary}

iex> req = ALLM.image_request("a watercolor kestrel", model: "gpt-image-1", size: {1024, 1024}, n: 2)
iex> :ok = ALLM.Validate.image_request(req)
iex> json = ALLM.Serializer.to_json!(req)
iex> {:ok, ^req} = ALLM.Serializer.from_json(json)
iex> {req.model, req.size, req.n}
{"gpt-image-1", {1024, 1024}, 2}

@spec json_schema(String.t(), map(), keyword()) :: map()

Build the canonical tagged map for a JSON-schema response format.

Returns %{type: :json_schema, name: name, schema: schema, strict: boolean}. :strict defaults to true; pass strict: false to relax provider-side schema enforcement.

Pass the returned map as :response_format on a request to ask the provider to constrain its output to the schema.

Atom-keyed schemas (and atom values such as type: :object) are normalized to strings via ALLM.JsonSchema.normalize/1, matching ALLM.Tool.new/1's :schema handling. Pre-stringified maps pass through verbatim (fast path).

Examples

iex> ALLM.json_schema("person", %{"type" => "object"})
%{type: :json_schema, name: "person", schema: %{"type" => "object"}, strict: true}

iex> ALLM.json_schema("person", %{"type" => "object"}, strict: false)
%{type: :json_schema, name: "person", schema: %{"type" => "object"}, strict: false}

iex> ALLM.json_schema("person", %{type: :object, properties: %{name: %{type: :string}}}).schema
%{"properties" => %{"name" => %{"type" => "string"}}, "type" => "object"}

@spec moderate(
  ALLM.Engine.t(),
  String.t() | [ALLM.ModerationRequest.item()] | ALLM.ModerationRequest.t(),
  keyword()
) ::
  {:ok, ALLM.ModerationResponse.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.ModerationAdapterError.t()}

Screen content for policy violations using the engine's :moderation_adapter.

Layer-C façade. Three input shapes:

• Binary — sugar for a one-element batch. Opts named in moderation_request/2 (:model, :options, :metadata) lift onto the built request; everything else is a call-control opt.
• List of items — each item a String.t/0 or an ALLM.ImagePart.t/0. Same opt-lifting rule.
• Pre-built %ALLM.ModerationRequest{} — dispatched verbatim; opts are NOT merged onto it.

The library does not decide what "unsafe" means. It returns the provider's own flagged boolean and the provider's per-category score map; there is no default threshold and no block?/2. Use ALLM.ModerationResponse.flagged?/1 for the provider's verdict, or ALLM.ModerationResult.score/2 to apply your own.

Result cardinality is type-dependent

This is the one surprising thing about the moderation wire, and it is a property of the provider endpoint rather than an ALLM choice:

• All-strings :input — a batch of length(input) independent items, so length(response.results) == length(request.input) and Enum.at(results, i) is the verdict for Enum.at(input, i).
• Any %ALLM.ImagePart{} present — the whole :input list is one multimodal item (text plus its images, judged together), so there is exactly one result, at index: 0.

ALLM.ModerationRequest.multimodal?/1 reports which shape a request is in, so the count is derivable before the call. The rule is stated normatively on ALLM.ModerationRequest.

Unknown opts — forwarded to the adapter

Any opt that is neither a request field nor a call-control opt is passed through to ALLM.ModerationAdapter.moderate/2 untouched, so provider-specific knobs need no façade change.

Adapter-presence gate

Returns {:error, %ALLM.Error.EngineError{reason: :no_moderation_adapter}} when engine.moderation_adapter == nil. This is the first gate — it fires ahead of validation and capability pre-flight, so a misconfigured engine never surfaces as a request problem.

Validation policy

Like embed/3 and unlike generate_image/3, this façade DOES call ALLM.Validate.moderation_request/1, returning {:error, %ALLM.Error.ValidationError{reason: :invalid_moderation_request}} on failure. An empty :input list and an empty-string item are both guaranteed provider rejections and should fail before the round-trip.

Batching — there is none

Unlike embed/3, moderate/3 does not chunk transparently, and the divergence is deliberate. A moderation call returns exactly one provider id per HTTP request, so merging N chunks would produce N ids with nowhere to put them; moderation is free, so the cost pressure that justifies chunking embeddings is absent. Adapters still declare ALLM.ModerationAdapter.max_batch_size/0 and reject an over-long input with %ALLM.Error.ModerationAdapterError{reason: :batch_too_large}, so the caller-side loop — for an all-strings :input — is:

adapter = engine.moderation_adapter

input
|> Enum.chunk_every(adapter.max_batch_size())
|> Enum.map(&ALLM.moderate(engine, &1))

Do not apply that loop to a multimodal :input. Per Result cardinality above, a list carrying an ALLM.ImagePart is one item judged as a whole, so there is nothing to chunk: splitting it would sever an image from the text it belongs to and turn one call into several with different verdict semantics. Gate on ALLM.ModerationRequest.multimodal?/1 if the shape is not known statically.

No :usage, but the span still carries the key

%ALLM.ModerationResponse{} has no :usage field — the endpoint is free and returns no usage object. The [:allm, :moderate, :stop] telemetry metadata nonetheless carries usage: nil unconditionally, so a metrics handler written against the :embed or :image span does not KeyError when pointed at this one. The :stop measurements result_count and flagged_count are likewise present on both the success and the error path (0 each on error).

:start and :stop metadata also carry input_count and multimodal. input_count is length(request.input) — the raw element count, not the provider's item count, which is 1 whenever multimodal is true (see Result cardinality above). The two agree exactly for an all-strings input, and multimodal rides alongside so a consumer can derive the item count without a second measurement — so a 40-element input carrying one ALLM.ImagePart reports input_count: 40 and is still under a ALLM.ModerationAdapter.max_batch_size/0 of 32.

Retry

:rate_limited, :provider_unavailable, :timeout and :network_error are retried under the engine's :retry policy; every other ALLM.Error.ModerationAdapterError reason surfaces immediately.

That budget is the façade's, applied once per call — moderate/3 does not chunk, so there is no per-chunk multiplier the way there is in embed/3. But opts[:retry] is also forwarded verbatim in the adapter's dispatch opts, so an adapter that runs its own ALLM.Retry.run/3 loop nests inside this one and the two budgets multiply for any reason both loops treat as retryable. With the default 3-attempt policy on each, a reason retryable at both layers costs up to 9 adapter calls and a reason retryable at only one costs up to 3. Read the adapter's own docs for which reasons it retries; embed/3 carries the same nesting and is the worked example.

Non-conforming adapters raise

ALLM.ModerationAdapter.moderate/2 must return {:ok, %ALLM.ModerationResponse{}} or {:error, %ALLM.Error.ModerationAdapterError{}} (invariant 2). Anything else raises ArgumentError naming the offending adapter rather than being laundered into this function's error union — the union above describes conforming adapters only. The published conformance suite deliberately does not bind that invariant, so this raise is what enforces it.

request_id precedence

opts[:request_id] wins over an auto-generated id. The id is forwarded to the adapter and filled onto response.request_id IFF the adapter left it nil; an adapter-populated id is preserved.

:stream opt is silently dropped

Moderation is request/response — there is no stream_moderate/3, by design. Passing stream: true does not error; the opt is ignored.

Examples

iex> engine = ALLM.Engine.new(
...> moderation_adapter: ALLM.Providers.FakeModeration,
...> adapter_opts: [moderation_script: [{:flagged, ["violence"]}]]
...>)
iex> {:ok, resp} = ALLM.moderate(engine, "…user text…")
iex> {ALLM.ModerationResponse.flagged?(resp),
...>  ALLM.ModerationResponse.flagged_categories(resp)}
{true, ["violence"]}

iex> engine = ALLM.Engine.new()
iex> {:error, %ALLM.Error.EngineError{reason: :no_moderation_adapter}} =
...> ALLM.moderate(engine, "is this ok?")
iex> :ok
:ok

@spec moderation_request(
  String.t() | [ALLM.ModerationRequest.item()],
  keyword()
) :: ALLM.ModerationRequest.t()

Build an %ALLM.ModerationRequest{} from a string or a list of items.

A bare string is normalised to a one-element list, so :input on the struct is always a list and no adapter or validator has to handle a union at the top level.

An item is a String.t/0 or an ALLM.ImagePart.t/0, and the element types decide the result cardinality — see the "Result cardinality" section of moderate/3, and ALLM.ModerationRequest for the normative statement.

Options

Only ALLM.ModerationRequest field names are read: :model, :options, :metadata. Every other key is ignored, which is what lets moderate/3 forward its own call-control opts (:request_id, :request_timeout, :retry, :adapter_opts, :api_key, :stream) through this function without them landing on the struct.

No validation runs here — call ALLM.Validate.moderation_request/1 if you want the field rules checked before dispatch. moderate/3 calls it for you.

Examples

iex> req = ALLM.moderation_request("is this ok?")
iex> req.input
["is this ok?"]
iex> req.model
nil

iex> req = ALLM.moderation_request(["one", "two"], model: "omni-moderation-latest")
iex> length(req.input)
2
iex> req.model
"omni-moderation-latest"

@spec request(
  [ALLM.Message.t()],
  keyword()
) :: ALLM.Request.t()

Build an %ALLM.Request{} from a list of messages and keyword opts. Delegates to ALLM.Request.new/2.

Does not validate — validation runs at the adapter boundary or via an explicit ALLM.Validate.request/1 call. Construction stays composable: request/2 returns a %Request{} directly, not an {:ok | :error} tuple.

Examples

iex> req = ALLM.request([ALLM.user("hi")])
iex> {length(req.messages), req.stream, req.tools}
{1, false, []}

iex> req = ALLM.request([ALLM.user("hi")], model: "gpt-4.1-mini", response_format: %{type: :json_object})
iex> {req.model, req.response_format}
{"gpt-4.1-mini", %{type: :json_object}}

@spec speech_request(
  String.t(),
  keyword()
) :: ALLM.SpeechRequest.t()

Build an %ALLM.SpeechRequest{} from the text to speak.

Options

Only ALLM.SpeechRequest field names are read: :model, :voice, :format, :instructions, :speed, :sample_rate, :options, :metadata. Every other key is ignored, which is what lets synthesize/3 forward its own call-control opts (:request_id, :request_timeout, :retry, :adapter_opts, :api_key, :stream) through this function without them landing on the struct.

No validation runs here — call ALLM.Validate.speech_request/1 if you want the field rules checked before dispatch. synthesize/3 calls it for you.

Examples

iex> req = ALLM.speech_request("Hello.", voice: "alloy", format: :mp3)
iex> {req.input, req.voice, req.format, req.model}
{"Hello.", "alloy", :mp3, nil}

@spec step(ALLM.Engine.t(), ALLM.Thread.t() | [ALLM.Message.t()], keyword()) ::
  {:ok, ALLM.StepResult.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.AdapterError.t()
     | ALLM.Error.ValidationError.t()}

Execute a single chat step (one adapter round-trip plus any auto-executed tool calls) and return a %ALLM.StepResult{}.

thread_or_messages is either an %ALLM.Thread{} or a list of %ALLM.Message{} (normalised via ALLM.Thread.from_messages/1). The thread is validated via ALLM.Validate.thread/1 at entry.

Use step/3 when you want a single round-trip — one adapter call, with any tool calls executed inline — but you don't need the multi-turn loop. For full multi-turn behaviour use chat/3.

Options

In addition to any provider-specific opts the adapter honours:

• :mode — :auto (default) executes tool calls inline; :manual returns them on the %StepResult{} for the caller to submit results.
• :tool_timeout — milliseconds per tool (default 30_000).
• :on_tool_error — :continue (default) or :halt.
• :tool_executor, :tool_result_encoder — module overrides.
• :prompt_cache — provider prompt caching; resolved and normalized exactly as for chat/3 (see "Prompt caching" there).
• Stream filter opts are accepted but have no effect on this non-streaming path.

Examples

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> adapter_opts: [
...> script: [
...> {:tool_call, id: "call_0", name: "weather", arguments: %{"city" => "NYC"}},
...> {:finish, :tool_calls}
...> ]
...> ],
...> tools: [ALLM.tool(
...> name: "weather",
...> description: "forecast by city",
...> schema: %{"type" => "object"},
...> handler: fn %{"city" => c} -> {:ok, %{forecast: "sunny", city: c}} end
...>)]
...>)
iex> {:ok, sr} = ALLM.step(engine, [ALLM.user("weather in NYC?")])
iex> {sr.done?, length(sr.tool_results)}
{false, 1}

@spec stream(ALLM.Engine.t(), ALLM.Thread.t() | [ALLM.Message.t()], keyword()) ::
  {:ok, Enumerable.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.AdapterError.t()
     | ALLM.Error.ValidationError.t()}

Stream a multi-turn chat loop as a lazy enumerable of ALLM.Event.t/0 values terminating in exactly one :chat_completed event.

thread_or_messages is either an %ALLM.Thread{} or a list of %ALLM.Message{}. The returned stream is open — no events fire until the caller reduces.

Single terminal :chat_completed

A naturally-terminating stream emits adapter events plus tool events for each turn, one :step_completed per turn, and exactly one trailing {:chat_completed, %{result: %ChatResult{}}} event. Both chat/3 and stream/3 |> ALLM.StreamCollector.to_chat_result/1 produce the SAME %ChatResult{} for identical inputs.

Consumer halts (Enum.take/2, Stream.take_while/2) produce NO :chat_completed event; callers needing a final %ChatResult{} for a cancelled stream collect events and call ALLM.StreamCollector.to_chat_result/1 on the partial state — the fallback path returns halted_reason: :cancelled.

Stream-first

chat/3 is itself a reducer over this stream. The streaming path is the primitive; the non-streaming variant exists so callers who don't need event-level visibility get a synchronous result.

Ask-user thread asymmetry

When a step's handler returns {:ask_user, _}, the streamed :step_completed.thread does NOT include the assistant question message — only the terminal :chat_completed.result.thread does. Consumers persisting thread state across turns must read ChatResult.thread, never :step_completed.thread.

:on_event scope

Same as chat/3 and stream_generate/3: :on_event observes only adapter-emitted events. Chat-layer events (:tool_execution_*, :tool_result_encoded, :ask_user_requested, :tool_halt, :step_completed, :chat_completed) are NOT delivered to :on_event.

Options

Same options as chat/3, including :prompt_cache (see "Prompt caching" there). The streaming filter opts (:emit_text_deltas, :emit_tool_deltas, :include_raw_chunks, :on_event) apply to each turn's adapter pass-through.

Examples

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> adapter_opts: [
...> scripts: [
...> [{:tool_call, id: "c0", name: "echo", arguments: %{"x" => 1}},
...> {:finish, :tool_calls}],
...> [{:text, "done"}, {:finish, :stop}]
...> ]
...> ],
...> tools: [ALLM.tool(
...> name: "echo",
...> description: "",
...> schema: %{},
...> handler: fn args -> {:ok, args} end
...>)]
...>)
iex> {:ok, stream} = ALLM.stream(engine, [ALLM.user("echo please")])
iex> events = Enum.to_list(stream)
iex> Enum.count(events, &match?({:chat_completed, _}, &1))
1

@spec stream_generate(ALLM.Engine.t(), ALLM.Request.t(), keyword()) ::
  {:ok, Enumerable.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.AdapterError.t()
     | ALLM.Error.ValidationError.t()}

Open a streaming generation against the engine's adapter.

Returns {:ok, enumerable} where the enumerable is a lazy stream of ALLM.Event.t/0 values (no event fires until the caller reduces), or {:error, struct} on a synchronous pre-flight failure (missing adapter, invalid request, adapter-reported pre-flight error).

Mid-stream adapter errors fold into a terminal :message_completed event with finish_reason: :error rather than a call-site error tuple — collect events with ALLM.StreamCollector.to_response/1 to recover the full %ALLM.Response{} (including metadata.error when populated).

Options

In addition to any provider-specific opts the adapter honours, the following streaming-layer keys are consumed by this function:

• :emit_text_deltas — true (default) keeps :text_delta events in the stream; false drops them. :text_completed and :message_completed are unaffected.
• :emit_tool_deltas — true (default) keeps :tool_call_delta events; false drops them.
• :include_raw_chunks — false (default) drops :raw_chunk events EXCEPT those with payload {:usage, _}, which always pass so %Response.usage can be populated downstream.
• :on_event — a 1-arity function invoked for every event BEFORE the filters apply. Exceptions raised inside the callback surface in the consumer's reducing process, not at this call site.

Multi-turn orchestration opts (:mode, :max_turns, :halt_when) are silently stripped — stream_generate/3 is single-request.

The request is caller-built, so prompt_cache: is not read from these opts: set ALLM.Request's :prompt_cache field on the request itself.

Examples

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> adapter_opts: [script: [{:text, "hi"}, {:finish, :stop}]]
...>)
iex> req = ALLM.request([ALLM.user("say hi")])
iex> {:ok, stream} = ALLM.stream_generate(engine, req)
iex> Enum.any?(Enum.to_list(stream), &match?({:message_completed, _}, &1))
true

@spec stream_step(ALLM.Engine.t(), ALLM.Thread.t() | [ALLM.Message.t()], keyword()) ::
  {:ok, Enumerable.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.AdapterError.t()
     | ALLM.Error.ValidationError.t()}

Execute a single chat step as a lazy stream of ALLM.Event.t/0 values.

thread_or_messages is either an %ALLM.Thread{} or a list of %ALLM.Message{}. The returned stream is open — no events fire until the caller reduces. Events arrive in this order: all adapter events (pass-through from stream_generate/3), then zero-to-N tool-execution event groups (per tool: :tool_execution_started → :tool_execution_completed → :tool_result_encoded / :ask_user_requested / :tool_halt), then exactly one terminal :step_completed event.

Options

Same as step/3, including :prompt_cache (see "Prompt caching" on chat/3). Additionally accepts the streaming filter opts (:emit_text_deltas, :emit_tool_deltas, :include_raw_chunks, :on_event) — they apply to the adapter-stream pass-through.

Examples

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> adapter_opts: [
...> script: [
...> {:tool_call, id: "call_0", name: "weather", arguments: %{"city" => "NYC"}},
...> {:finish, :tool_calls}
...> ]
...> ],
...> tools: [ALLM.tool(
...> name: "weather",
...> description: "forecast by city",
...> schema: %{"type" => "object"},
...> handler: fn %{"city" => c} -> {:ok, %{forecast: "sunny", city: c}} end
...>)]
...>)
iex> {:ok, stream} = ALLM.stream_step(engine, [ALLM.user("weather in NYC?")])
iex> events = Enum.to_list(stream)
iex> Enum.any?(events, &match?({:step_completed, _}, &1))
true

@spec stream_synthesize(
  ALLM.Engine.t(),
  String.t() | ALLM.SpeechRequest.t(),
  keyword()
) ::
  {:ok, Enumerable.t(ALLM.SpeechEvent.t())}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.SpeechAdapterError.t()}

Stream speech for a whole text using the engine's :speech_adapter.

Layer-C façade. Returns {:ok, events}, a lazy enumerable of ALLM.SpeechEvent.t/0: :speech_started, one or more :audio_delta events carrying the next bytes of audio, then :speech_completed. Play or forward each delta as it arrives; ALLM.AudioStream.collect_speech/1 folds the stream into the ALLM.SpeechResponse synthesize/3 returns.

Streaming uses the same :speech_adapter slot as synthesize/3. The adapter opts in by also implementing ALLM.SpeechStreamAdapter.

Input shapes

The same two as synthesize/3: a binary, with the speech_request/2 opts lifted onto the built request, or a pre-built %ALLM.SpeechRequest{}, dispatched verbatim.

Gate order

Each gate returns synchronously, before any I/O:

1. {:error, %ALLM.Error.EngineError{reason: :no_speech_adapter}} when engine.speech_adapter == nil.
2. {:error, %ALLM.Error.EngineError{reason: :missing_stream_adapter}} when the slot's adapter does not export ALLM.SpeechStreamAdapter.stream_synthesize/2.
3. ALLM.Validate.speech_request/1, returning {:error, %ALLM.Error.ValidationError{reason: :invalid_speech_request}}.
4. Dispatch. The adapter runs its own gates and may return {:error, %ALLM.Error.SpeechAdapterError{}} before opening anything.

Model resolution

As synthesize/3: request.model, else engine.speech_model, else the adapter's default. engine.model is never consulted.

No retry after open

Streams are not retried. A synchronous {:error, _} from the adapter is returned as-is, and an error after the stream opened ends the stream with {:error, %ALLM.Error.SpeechAdapterError{}} — nothing follows it. Retry by calling again.

Laziness and halting

No audio is requested until the enumerable is reduced. Halting it early (Enum.take/2, a Stream.take_while/2 that stops) releases the adapter's connection. opts[:stream_timeout] (milliseconds of silence, default 60,000) is forwarded to the adapter.

Time to first audio

[:allm, :audio, :first_chunk] fires once per stream, at the first :audio_delta, with measurements.latency in native time units measured from this call. See ALLM.Telemetry.

Telemetry carries no audio

The call runs inside an [:allm, :stream_synthesize, …] span. It stops when the enumerable is returned, so its :stop metadata has response: nil, and neither it nor the first-chunk event carries audio.

Non-conforming adapters raise

An adapter that returns anything but {:ok, enumerable} or {:error, %ALLM.Error.SpeechAdapterError{}} raises ArgumentError naming it. So does an element of the stream that is not a ALLM.SpeechEvent.t/0, when it is reduced.

Examples

iex> engine = ALLM.Engine.new(speech_adapter: ALLM.Providers.FakeSpeech)
iex> {:ok, events} = ALLM.stream_synthesize(engine, "Hello.", format: :pcm)
iex> {:ok, resp} = ALLM.AudioStream.collect_speech(events)
iex> {ALLM.Audio.to_binary(resp.audio), resp.format}
{{:ok, "FAKE-AUDIO:Hello."}, :pcm}

iex> {:error, %ALLM.Error.EngineError{reason: :no_speech_adapter}} =
...> ALLM.stream_synthesize(ALLM.Engine.new(), "Hello.")
iex> :ok
:ok

@spec stream_synthesize_input(ALLM.Engine.t(), Enumerable.t(String.t()), keyword()) ::
  {:ok, Enumerable.t(ALLM.SpeechEvent.t())}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.SpeechAdapterError.t()}

Stream speech for text that arrives in chunks, using the engine's :speech_adapter.

Layer-C façade. input is an enumerable of strings, typically an LLM's output as it is generated:

{:ok, chat} = ALLM.stream(engine, [ALLM.user("Tell me a story.")])
{:ok, audio} = ALLM.stream_synthesize_input(engine, ALLM.AudioStream.text_deltas(chat))

Returns {:ok, events} with the same ALLM.SpeechEvent.t/0 grammar as stream_synthesize/3. Audio for the first words can arrive before the input has finished.

Input shapes

The request is built with speech_request("", opts), so the same request-field opts apply, or taken from opts[:request], a pre-built %ALLM.SpeechRequest{} that is authoritative (other request-field opts are ignored, and its :input is not used). Any other opts[:request] value raises ArgumentError before the span opens.

The adapter reduces input in a separate process, so an input that reads the calling process's mailbox or process dictionary must be relayed. An empty string chunk is skipped. A chunk that is not a UTF-8 string ends the stream with metadata.cause: :invalid_input_chunk, and an input that raises ends it with metadata.cause: :input_raised (see ALLM.AudioStream).

Gate order

Each gate returns synchronously, before input is touched:

1. {:error, %ALLM.Error.EngineError{reason: :no_speech_adapter}} when engine.speech_adapter == nil.
2. {:error, %ALLM.Error.EngineError{reason: :missing_stream_adapter}} when the adapter does not export ALLM.SpeechStreamAdapter.stream_synthesize_input/3 — including an adapter that streams whole texts only.
3. {:error, %ALLM.Error.ValidationError{reason: :invalid_speech_request}} with {:input, :invalid_shape} when input is a binary or not an enumerable.
4. ALLM.Validate.speech_request(request, input: :streamed), which checks every field except :input.
5. Dispatch; the adapter's own gates may return {:error, %ALLM.Error.SpeechAdapterError{}}.

Model resolution, retry, halting, telemetry

As stream_synthesize/3: engine.speech_model fills a nil request.model; nothing is retried; halting releases the connection and stops the input's reduction; [:allm, :audio, :first_chunk] fires at the first :audio_delta. The span is [:allm, :stream_synthesize, …] with input_length: nil, and carries no audio.

Examples

iex> engine = ALLM.Engine.new(speech_adapter: ALLM.Providers.FakeSpeech)
iex> {:ok, events} = ALLM.stream_synthesize_input(engine, ["Hel", "", "lo."])
iex> for {:audio_delta, bytes} <- events, do: bytes
["FAKE-AUDIO:Hel", "FAKE-AUDIO:lo."]

@spec stream_transcribe(ALLM.Engine.t(), Enumerable.t(binary() | :commit), keyword()) ::
  {:ok, Enumerable.t(ALLM.TranscriptionEvent.t())}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.TranscriptionAdapterError.t()}

Transcribe audio as it arrives, using the engine's :transcription_adapter.

Layer-C façade. input is an enumerable of PCM16 little-endian mono audio chunks at :sample_rate (default 16,000 Hz), and the atom :commit to force a segment boundary. Returns {:ok, events}, a lazy enumerable of ALLM.TranscriptionEvent.t/0: :transcription_started, then :partial_transcript events (each replaces the previous partial) and :committed_transcript segments (final, appended), then :transcription_completed, whose :text is the whole transcript. ALLM.AudioStream.collect_transcription/1 folds the stream into an ALLM.TranscriptionResponse.

The adapter opts in by implementing ALLM.TranscriptionStreamAdapter alongside ALLM.TranscriptionAdapter; check ALLM.TranscriptionStreamAdapter.stream_sample_rates/0 before opening a microphone.

Input shapes

The request is built from the ALLM.TranscriptionStreamRequest field opts (:model, :language, :sample_rate, :commit_strategy, :timestamps, :logprobs, :options, :metadata), or taken from opts[:request], a pre-built %ALLM.TranscriptionStreamRequest{} that is authoritative. Any other opts[:request] value (a batch %ALLM.TranscriptionRequest{}, say) raises ArgumentError before the span opens.

The adapter reduces input in a separate process, so an input that reads the calling process's mailbox must be relayed. Chunk boundaries are free: a chunk may split a sample. An element that is neither a binary nor :commit, or an input whose total length is odd, ends the stream with metadata.cause: :invalid_input_chunk.

Gate order

Each gate returns synchronously, before input is touched:

1. {:error, %ALLM.Error.EngineError{reason: :no_transcription_adapter}} when engine.transcription_adapter == nil.
2. {:error, %ALLM.Error.EngineError{reason: :missing_stream_adapter}} when the adapter does not export ALLM.TranscriptionStreamAdapter.stream_transcribe/3.
3. {:error, %ALLM.Error.ValidationError{reason: :invalid_transcription_request}} with {:input, :invalid_shape} when input is a binary or not an enumerable.
4. ALLM.Validate.transcription_stream_request/1.
5. Dispatch; the adapter's own gates (an unsupported :sample_rate, …) may return {:error, %ALLM.Error.TranscriptionAdapterError{}}.

Timings and log-probabilities

timestamps: true and logprobs: true work as in transcribe/3, and the spans are ALLM.TranscriptSpan structs. With either flag set, each :committed_transcript payload carries a :spans key (a list, or nil when the adapter has no trustworthy span data for that segment), and :transcription_completed carries :spans, every committed segment's spans in order ([] when nothing was committed), which ALLM.AudioStream.collect_transcription/1 copies onto the response. With both flags false, neither event has a :spans key. On a realtime provider a flag can delay each committed segment while the adapter waits for its timing data; see the adapter's docs.

Model resolution

request.model, else the adapter's own realtime default. engine.transcription_model is not used: a provider's batch and realtime transcription models are different models, so an engine configured for transcribe/3 would break this call.

Retry, halting, telemetry

Streams are not retried, and an error after the stream opened ends it with {:error, %ALLM.Error.TranscriptionAdapterError{}}. Halting the enumerable closes the connection and stops the input's reduction. opts[:stream_timeout] (default 60,000 ms of silence) is forwarded to the adapter. The call runs inside an [:allm, :stream_transcribe, …] span whose :stop has response: nil, and [:allm, :audio, :first_chunk] fires at the first :partial_transcript, or at the first non-blank :committed_transcript when a provider commits with no partial before it. Neither carries audio or transcript text.

Examples

iex> engine = ALLM.Engine.new(
...> transcription_adapter: ALLM.Providers.FakeTranscription,
...> adapter_opts: [transcription_script: [{:ok, "hello there"}]]
...>)
iex> pcm = :binary.copy(<<0>>, 3_200)
iex> {:ok, events} = ALLM.stream_transcribe(engine, [pcm], sample_rate: 16_000)
iex> {:ok, resp} = ALLM.AudioStream.collect_transcription(events)
iex> resp.text
"hello there"

@spec synthesize(ALLM.Engine.t(), String.t() | ALLM.SpeechRequest.t(), keyword()) ::
  {:ok, ALLM.SpeechResponse.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.SpeechAdapterError.t()}

Turn text into speech using the engine's :speech_adapter.

Layer-C façade. Two input shapes:

• Binary — the text to speak. Opts named in speech_request/2 (:model, :voice, :format, :instructions, :speed, :sample_rate, :options, :metadata) lift onto the built request; everything else is a call-control opt.
• Pre-built %ALLM.SpeechRequest{} — dispatched verbatim; opts are NOT merged onto it.

On success response.audio is an ALLM.Audio holding the bytes, and response.format says which file format arrived (see ALLM.SpeechResponse). ALLM.Audio.to_binary/1 returns the bytes.

Model resolution

The model comes from the speech slot, never from the chat model: request.model, else engine.speech_model, else the adapter's own default. engine.model is not consulted — a chat model name is not a speech model name on any provider. On the binary shape opts[:model] sets request.model; a pre-built request's :model is authoritative.

Gate order

1. {:error, %ALLM.Error.EngineError{reason: :no_speech_adapter}} when engine.speech_adapter == nil — first, so a misconfigured engine never surfaces as a request problem.
2. ALLM.Validate.speech_request/1, returning {:error, %ALLM.Error.ValidationError{reason: :invalid_speech_request}} on failure (an empty :input, an unknown :format, …).
3. Dispatch to ALLM.SpeechAdapter.synthesize/2, under the retry policy below.

Unknown opts — forwarded to the adapter

Any opt that is neither a request field nor a call-control opt is passed through to ALLM.SpeechAdapter.synthesize/2 untouched, so provider-specific knobs need no façade change.

Retry

:rate_limited, :provider_unavailable, :timeout and :network_error are retried under the engine's :retry policy; every other ALLM.Error.SpeechAdapterError reason surfaces immediately.

opts[:retry] is also forwarded to the adapter, and a speech adapter may run its own retry loop inside this one. The two budgets then multiply: with the default 3-attempt policy on each, a reason retryable at both layers costs up to 9 synthesis calls. Read the adapter's own docs for which reasons it retries.

Telemetry carries the audio

The call runs inside an [:allm, :synthesize, …] span (see ALLM.Telemetry). Its :stop metadata includes the whole %ALLM.SpeechResponse{}, and with it the audio bytes, so a handler that logs or ships metadata.response wholesale moves the full payload on every call. The audio_bytes measurement gives the size without touching it.

Non-conforming adapters raise

ALLM.SpeechAdapter.synthesize/2 must return {:ok, %ALLM.SpeechResponse{}} or {:error, %ALLM.Error.SpeechAdapterError{}} (invariant 1). Anything else raises ArgumentError naming the offending adapter rather than being laundered into this function's error union.

request_id precedence

opts[:request_id] wins over an auto-generated id. The id is forwarded to the adapter and filled onto response.request_id IFF the adapter left it nil; an adapter-populated id is preserved.

Streaming

Synthesis here is request/response: the whole clip arrives in one response, and passing stream: true does not error — the opt is ignored. To play audio while it is still being synthesized, call stream_synthesize/3 (whole text) or stream_synthesize_input/3 (text that arrives in chunks) instead.

Examples

iex> engine = ALLM.Engine.new(speech_adapter: ALLM.Providers.FakeSpeech)
iex> {:ok, resp} = ALLM.synthesize(engine, "Hello.", format: :wav)
iex> {ALLM.Audio.to_binary(resp.audio), resp.format}
{{:ok, "FAKE-AUDIO:Hello."}, :wav}

iex> {:error, %ALLM.Error.EngineError{reason: :no_speech_adapter}} =
...> ALLM.synthesize(ALLM.Engine.new(), "Hello.")
iex> :ok
:ok

@spec system(String.t()) :: ALLM.Message.t()

Build a system-role %ALLM.Message{} from a text string.

Examples

iex> ALLM.system("be helpful")
%ALLM.Message{role: :system, content: "be helpful", name: nil, tool_call_id: nil, metadata: %{}}

@spec tool(keyword()) :: ALLM.Tool.t()

Build an %ALLM.Tool{} from keyword opts. Delegates to ALLM.Tool.new/1.

:name, :description, and :schema are required; omitting any raises ArgumentError. :handler is optional. Pass manual: true to opt this tool out of automatic execution under chat/3's mode: :auto. Pass compact: true to send the tool to the model as a one-line stub whose full definition the model reads on demand through a tool_help tool, and :summary to override that stub's one-line summary; see ALLM.Tool and ALLM.ToolHelp.

Examples

iex> tool = ALLM.tool(name: "weather", description: "weather by city", schema: %{"type" => "object"})
iex> {tool.name, tool.description}
{"weather", "weather by city"}

@spec tool_result(String.t(), String.t() | map()) :: ALLM.Message.t()

Build a tool-role %ALLM.Message{} carrying a tool-call result.

tool_call_id must match the :id of the ALLM.ToolCall that produced this result so the provider can match results to calls. content is either a binary or a JSON-serializable map.

Examples

iex> msg = ALLM.tool_result("call_abc", %{ok: true})
iex> {msg.role, msg.tool_call_id, msg.content}
{:tool, "call_abc", %{ok: true}}

@spec transcribe(
  ALLM.Engine.t(),
  ALLM.Audio.t() | ALLM.TranscriptionRequest.t(),
  keyword()
) ::
  {:ok, ALLM.TranscriptionResponse.t()}
  | {:error,
     ALLM.Error.EngineError.t()
     | ALLM.Error.ValidationError.t()
     | ALLM.Error.TranscriptionAdapterError.t()}

Turn speech into text using the engine's :transcription_adapter.

Layer-C façade. Two input shapes:

• %ALLM.Audio{} — the clip to transcribe. Opts named in transcription_request/2 (:model, :language, :prompt, :options, :metadata) lift onto the built request; everything else is a call-control opt.
• Pre-built %ALLM.TranscriptionRequest{} — dispatched verbatim; opts are NOT merged onto it.

On success response.text is the transcript (possibly "" for silence). response.usage is always an ALLM.Usage; providers that bill by the second report response.duration_seconds instead of tokens.

Model resolution

The model comes from the transcription slot, never from the chat model: request.model, else engine.transcription_model, else the adapter's own default. engine.model is not consulted. On the %Audio{} shape opts[:model] sets request.model; a pre-built request's :model is authoritative.

Gate order

1. {:error, %ALLM.Error.EngineError{reason: :no_transcription_adapter}} when engine.transcription_adapter == nil.
2. ALLM.Validate.transcription_request/1, returning {:error, %ALLM.Error.ValidationError{reason: :invalid_transcription_request}} on failure (an :audio that is not an ALLM.Audio, …).
3. Dispatch to ALLM.TranscriptionAdapter.transcribe/2. The adapter runs its own gates before any upload — unresolvable audio, audio over ALLM.TranscriptionAdapter.max_audio_bytes/0, an unsupported MIME type — and returns them as %ALLM.Error.TranscriptionAdapterError{reason: :invalid_request}.

Unknown opts — forwarded to the adapter

Any opt that is neither a request field nor a call-control opt is passed through to ALLM.TranscriptionAdapter.transcribe/2 untouched.

Retry

:rate_limited, :provider_unavailable, :timeout and :network_error are retried under the engine's :retry policy; every other ALLM.Error.TranscriptionAdapterError reason (including :content_filter) surfaces immediately.

This is the only retry loop for transcription. The bundled transcription adapters do not retry on their own, because each attempt re-uploads the whole clip — so with the default 3-attempt policy a clip is uploaded at most 3 times. A third-party adapter that runs its own loop would multiply that budget; calling an adapter's transcribe/2 directly gets no retries at all.

Non-conforming adapters raise

ALLM.TranscriptionAdapter.transcribe/2 must return {:ok, %ALLM.TranscriptionResponse{}} or {:error, %ALLM.Error.TranscriptionAdapterError{}} (invariant 1). Anything else raises ArgumentError naming the offending adapter.

request_id precedence

opts[:request_id] wins over an auto-generated id; it is forwarded to the adapter and filled onto response.request_id IFF the adapter left it nil.

Streaming

Transcription here is request/response: the transcript arrives in one response, and stream: true is ignored. To transcribe audio while it is still arriving (a microphone, a call), use stream_transcribe/3.

Timings and log-probabilities

timestamps: true asks for per-word start and end times and logprobs: true for per-word or per-token log-probabilities. Both land on response.spans, a list of ALLM.TranscriptSpan structs, and ALLM.TranscriptionResponse.mean_logprob/1 averages the spoken units. With both flags false, response.spans is nil. An attribute that was not asked for is nil on every span even when the provider sent it. An adapter that cannot honour a true flag refuses with %ALLM.Error.TranscriptionAdapterError{reason: :unsupported_feature} before any upload. If the provider's response carries no span data, a blank transcript succeeds with spans: [], and any other transcript fails with :unsupported_feature, metadata.cause: :absent_from_response and the transcript on metadata.text.

Examples

iex> engine = ALLM.Engine.new(
...> transcription_adapter: ALLM.Providers.FakeTranscription,
...> adapter_opts: [transcription_script: [{:ok, "hello there"}]]
...>)
iex> audio = ALLM.Audio.from_binary("ID3…", "audio/mpeg")
iex> {:ok, %ALLM.TranscriptionResponse{text: text}} = ALLM.transcribe(engine, audio)
iex> text
"hello there"

iex> engine = ALLM.Engine.new(
...> transcription_adapter: ALLM.Providers.FakeTranscription,
...> adapter_opts: [transcription_script: [{:ok, "hello there"}]]
...>)
iex> audio = ALLM.Audio.from_binary("ID3…", "audio/mpeg")
iex> {:ok, response} = ALLM.transcribe(engine, audio, timestamps: true, logprobs: true)
iex> for span <- response.spans, do: {span.text, span.end_seconds, span.logprob}
[{"hello", 0.5, -0.1}, {"there", 1.0, -0.1}]
iex> ALLM.TranscriptionResponse.mean_logprob(response)
-0.1

iex> {:error, %ALLM.Error.EngineError{reason: :no_transcription_adapter}} =
...> ALLM.transcribe(ALLM.Engine.new(), ALLM.Audio.from_binary("ID3…", "audio/mpeg"))
iex> :ok
:ok

@spec transcription_request(
  ALLM.Audio.t(),
  keyword()
) :: ALLM.TranscriptionRequest.t()

Build an %ALLM.TranscriptionRequest{} from an ALLM.Audio value.

Options

Only ALLM.TranscriptionRequest field names are read: :model, :language, :prompt, :timestamps, :logprobs, :options, :metadata. Every other key is ignored, so transcribe/3 can forward its call-control opts through this function.

No validation runs here — transcribe/3 calls ALLM.Validate.transcription_request/1 for you. Building a request never reads the audio: ALLM.Audio.from_file/1 stores only the path.

Examples

iex> req = ALLM.transcription_request(ALLM.Audio.from_file("clip.mp3"), language: "en")
iex> {req.audio.mime_type, req.language, req.model}
{"audio/mpeg", "en", nil}

@spec unwrap({:ok, ALLM.Response.t()} | {:error, term()}) ::
  {:ok, String.t()} | {:error, term()}

Fold a generate/3-shaped return tuple into {:ok, text} | {:error, _}.

Useful when the caller just wants the response text or a clear error and doesn't need the full %Response{}. Composes with the pipe-into-generate/3 pattern:

engine
|> ALLM.generate(ALLM.request([ALLM.user("hi")]))
|> ALLM.unwrap()

Clauses

• {:ok, %Response{finish_reason: :stop, message: %Message{content: list}}} where list is a list (vision / structured parts) → {:error, :structured_content}. The caller should access :message directly. This branch fires BEFORE the text fold below.
• {:ok, %Response{finish_reason: :stop}} → delegates to ALLM.Response.text/1 (which prefers :output_text over message.content). Returns {:ok, text} when text is a binary; {:error, :empty_stop_response} when both :output_text and message.content are absent / non-binary.
• {:ok, %Response{finish_reason: :error, metadata: %{error: e}}} → {:error, e} (mid-stream error folded back to the call site).
• {:ok, %Response{finish_reason: other}} → {:error, {:non_stop_finish, other}} for non-stop finishes (:length, :tool_calls, :content_filter, :other).
• {:error, _} = err → err (pass-through).

Examples

iex> engine = ALLM.Engine.new(
...> adapter: ALLM.Providers.Fake,
...> adapter_opts: [script: [{:text, "hello"}, {:finish, :stop}]]
...>)
iex> ALLM.unwrap(ALLM.generate(engine, ALLM.request([ALLM.user("hi")])))
{:ok, "hello"}

@spec user(String.t()) :: ALLM.Message.t()

Build a user-role %ALLM.Message{} from a text string.

Examples

iex> ALLM.user("hi")
%ALLM.Message{role: :user, content: "hi", name: nil, tool_call_id: nil, metadata: %{}}