Arcana.LLM protocol (Arcana v3.0.0)

Copy Markdown View Source

Protocol for LLM adapters used by Arcana.

Arcana accepts any LLM that implements this protocol. Built-in implementations:

  • Model strings via Req.LLM (e.g., "openai:gpt-4o-mini", "zai:glm-4.5-flash")
  • Tuples of {model_string, opts} for passing options like :api_key
  • Tuples of {module, function} naming a function of arity 1, 2, or 3 (prompt, prompt, context, or prompt, context, opts)
  • Anonymous functions (for testing)

Prefer the {module, function} form for config :arcana, :llm — unlike a captured function, it serializes into a release's sys.config, and it works without req_llm.

Examples

# Model string (requires req_llm)
Arcana.ask("question", llm: "openai:gpt-4o-mini", repo: MyApp.Repo)

# With options
Arcana.ask("question", llm: {"zai:glm-4.7", api_key: "key"}, repo: MyApp.Repo)

# Module/function tuple (release-safe for config)
config :arcana, llm: {MyApp.LLM, :complete}

# Function (for testing)
Arcana.ask("question", llm: fn _prompt -> {:ok, "answer"} end, repo: MyApp.Repo)

Summary

Types

t()

All the types that implement this protocol.

Functions

Completes a prompt with the given context and options.

Types

t()

@type t() :: term()

All the types that implement this protocol.

Functions

complete(llm, prompt, context, opts)

Completes a prompt with the given context and options.

Returns {:ok, response} or {:error, reason}.