ALLM.Request (allm v0.6.1)

Copy Markdown View Source

A single LLM request — Layer A serializable data.

Carries the :messages list, optional :model, :tools, :tool_choice, :temperature, :max_tokens, :response_format, and adapter-opaque :options and :metadata.

Fields

FieldTypeDefaultNotes
:messages[%Message{}](required)The conversation.
:modelString.t | nilnilLate-resolved against the engine.
:tools[%Tool{}][]Declared tools.
:tool_choice:auto | :none | :required | String.t | map | nilnilProvider-specific shapes pass through verbatim.
:temperaturenumber | nilnil
:max_tokensnon_neg_integer | nilnilAnthropic's adapter injects 1024 if nil.
:response_formatnil | :text | %{type: :json_object} | %{type: :json_schema, ...}nilBuild with ALLM.json_schema/3.
:streambooleanfalse
:structured_finalizebooleanfalseSynthetic-tool fallback for providers without native JSON-schema mode.
:prompt_cachenil | %{key: String.t | nil, retention: :short | :long}nilProvider-neutral prompt-cache request. See "Prompt caching".
:optionsmap%{}Adapter-opaque pass-through.
:metadatamap%{}Caller-owned.

Validation lives in ALLM.Validate.request/1. new/2 itself does not validate — it stays composable so callers opt into validation explicitly.

Prompt caching

:prompt_cache asks the provider to cache the request's prompt prefix. It is nil (no request) or a map with exactly two keys:

  • :key — a routing key some providers use to send related requests to the same cache, or nil. Sent verbatim; never hashed or transformed.
  • :retention — :short keeps the provider's default cache lifetime and sends no lifetime field. :long asks for the longest per-request lifetime the provider offers.

ALLM.Validate.request/1 rejects any other shape with {:prompt_cache, :invalid_shape}. The field is typed rather than an :options entry so it has one validated shape and survives a JSON round-trip with its atoms restored. Raw provider cache parameters placed in :options are still sent as-is and take precedence over what an adapter derives from this field.

iex> req = ALLM.Request.new([%ALLM.Message{role: :user, content: "hi"}],
...>   prompt_cache: %{key: "recipe-42", retention: :long})
iex> ALLM.Validate.request(req)
:ok
iex> {:ok, decoded} = ALLM.Serializer.from_json(ALLM.Serializer.to_json!(req))
iex> decoded.prompt_cache
%{key: "recipe-42", retention: :long}

Round-trip

iex> req = ALLM.Request.new([%ALLM.Message{role: :user, content: "hi"}], model: "fake:x")
iex> json = ALLM.Serializer.to_json!(req)
iex> {:ok, ^req} = ALLM.Serializer.from_json(json)
iex> req.model
"fake:x"

See also guides/getting_started.md.

Summary

Types

prompt_cache()
response_format()
t()
tool_choice()

Functions

new(messages, opts \\ [])

Build a %Request{} from a list of messages and keyword opts.

Types

prompt_cache()

@type prompt_cache() :: nil | %{key: String.t() | nil, retention: :short | :long}

response_format()

@type response_format() ::
  nil
  | :text
  | %{type: :json_object}
  | %{type: :json_schema, name: String.t(), schema: map(), strict: boolean()}

t()

@type t() :: %ALLM.Request{
  max_tokens: non_neg_integer() | nil,
  messages: [ALLM.Message.t()],
  metadata: map(),
  model: String.t() | nil,
  options: map(),
  prompt_cache: prompt_cache(),
  response_format: response_format(),
  stream: boolean(),
  structured_finalize: boolean(),
  temperature: number() | nil,
  tool_choice: tool_choice(),
  tools: [ALLM.Tool.t()]
}

tool_choice()

@type tool_choice() :: :auto | :none | :required | String.t() | map() | nil

Functions

new(messages, opts \\ [])

@spec new(
  [ALLM.Message.t()],
  keyword()
) :: t()

Build a %Request{} from a list of messages and keyword opts.

messages is required and becomes the :messages field. opts may set any other struct field; unknown keys raise ArgumentError via struct!/2.

Examples

iex> req = ALLM.Request.new([%ALLM.Message{role: :user, content: "hi"}])
iex> req.stream
false
iex> length(req.messages)
1

iex> req = ALLM.Request.new([%ALLM.Message{role: :user, content: "hi"}], model: "fake:x", temperature: 0.2)
iex> {req.model, req.temperature}
{"fake:x", 0.2}