Candil.Structured (Candil v3.0.0)

Copy Markdown View Source

Force the model to produce JSON matching a schema.

Wraps Candil.Backend.chat/3 with:

  1. A system prompt that tells the model to respond with JSON only.
  2. A post-validation step that re-parses the response and rejects anything that doesn't match the schema (best-effort: required properties present and primitive types correct).

On validation failure, retries once with a feedback message describing the mismatch.

Example

schema = %{
  "type" => "object",
  "properties" => %{
    "title" => %{"type" => "string"},
    "year" => %{"type" => "integer"}
  },
  "required" => ["title", "year"]
}

Candil.Structured.complete(:llama3, "Recommend a sci-fi book", schema)
# => {:ok, %{"title" => "Dune", "year" => 1965}}

Summary

Types

result()
schema()

Functions

complete(model, prompt, schema, opts \\ [])

Ask the model to complete a prompt with output that conforms to the given JSON schema. Returns the parsed map on success.

Types

result()

@type result() :: {:ok, map()} | {:error, term()}

schema()

@type schema() :: map()

Functions

complete(model, prompt, schema, opts \\ [])

@spec complete(String.t() | atom(), String.t(), schema(), keyword()) :: result()

Ask the model to complete a prompt with output that conforms to the given JSON schema. Returns the parsed map on success.

Options

  • :backend — explicit Candil.Backend module to use. Defaults to Candil.Backend.for(Backend.infer_provider(:local, model), model).
  • :max_retries — number of retry attempts after validation failure (default 1; total attempts = 1 + max_retries).
  • :temperature — passed through to the backend.