Structured Output

Structured output asks the model to return valid JSON matching a schema you provide. The decoded map is available in response.parsed — no regex, no manual Jason.decode/1.

Basic example

schema = %{
  "type" => "object",
  "properties" => %{
    "name"  => %{"type" => "string"},
    "age"   => %{"type" => "integer"},
    "email" => %{"type" => "string"}
  },
  "required" => ["name", "age", "email"]
}

{:ok, response} = LLM.generate("Extract the person: Alice is 30, alice@example.com",
  provider: :openai,
  model: "gpt-4o",
  structured_output: schema
)

response.parsed
#=> %{"name" => "Alice", "age" => 30, "email" => "alice@example.com"}

Schema format

Bare schema (name defaults to "output")

structured_output: %{"type" => "object", "properties" => %{...}}

Named schema

Wrap with %{name: ..., schema: ...} to give the schema an explicit name. Some providers surface the name in their APIs or logs:

structured_output: %{
  name: "person",
  schema: %{
    "type" => "object",
    "properties" => %{
      "name" => %{"type" => "string"},
      "age"  => %{"type" => "integer"}
    },
    "required" => ["name", "age"]
  }
}

The response.parsed field

response.parsed holds the decoded map when the model returns valid JSON matching your schema. It is nil if the model did not return parseable JSON (see Error handling below).

{:ok, response} = LLM.generate(prompt, provider: :openai, model: "gpt-4o", structured_output: schema)

case response.parsed do
  nil  -> {:error, :no_structured_output}
  data -> {:ok, data}
end

response.message.content still contains the raw text or tool-call content for debugging.

Provider notes

OpenAI (/v1/chat/completions)

Encodes as response_format: {type: "json_schema", json_schema: {name, schema}}. Requires additionalProperties: false in nested objects for strict mode — add it explicitly if your model complains:

schema = %{
  "type" => "object",
  "additionalProperties" => false,
  "properties" => %{
    "title"   => %{"type" => "string"},
    "summary" => %{"type" => "string"}
  },
  "required" => ["title", "summary"]
}

OpenAI Responses (/v1/responses)

Encodes as text: {format: {type: "json_schema", name: ..., schema: ...}}.

Anthropic

Uses tool-forcing: the library injects a hidden __structured_output__ tool with your schema and sets tool_choice: {type: "tool", name: "__structured_output__"}. The tool call is never executed — the model's JSON arguments are extracted directly into response.parsed. auto_tools is set to false automatically so your own tools are not executed mid-request.

This means structured output and user-defined tools cannot be combined on Anthropic in a single call.

Gemini

Sets generationConfig.responseMimeType: "application/json" and responseSchema in the request. Gemini rejects additionalProperties — omit it from your schema when targeting Gemini:

# Works on Gemini (no additionalProperties)
schema = %{
  "type" => "object",
  "properties" => %{
    "sentiment" => %{"type" => "string", "enum" => ["positive", "negative", "neutral"]}
  },
  "required" => ["sentiment"]
}

Structured output vs tools

Use structured output to extract data, classify text, or generate typed objects. Use tools to give the model capabilities (web search, database access, calculations).

Error handling

response.parsed is nil when:

• The model returned plain text instead of JSON (common with weaker models or ambiguous prompts)
• The JSON was valid but did not decode to a map (e.g., the model returned a bare array)
• A provider error occurred upstream

{:ok, response} = LLM.generate(prompt,
  provider: :anthropic,
  model: "claude-opus-4-5",
  structured_output: schema
)

case response.parsed do
  %{} = data ->
    process(data)

  nil ->
    # Fallback: try to extract from raw content
    Logger.warning("Structured output missing, raw: #{inspect(response.message.content)}")
    {:error, :parse_failed}
end

Tips for reliable parsing:

• Keep schemas simple — avoid deeply nested oneOf/anyOf with weaker models
• Make all required fields explicit in "required"
• On Gemini, omit additionalProperties
• On OpenAI strict mode, add "additionalProperties" => false to every object

See also

• Tools — define tools the model can call
• Providers — provider-specific options and capabilities