Spectre.LLM behaviour (Spectre v0.3.0)

Small LLM boundary used by Spectre.Runner.

The runner sends a typed Spectre.Prompt.Plan here. A module that exports complete_plan/2 receives that plan with separate instruction, context, and task fragments. Existing modules, tuples, and functions continue to receive the plan's legacy string byte-for-byte. Tuple/function adapters may opt into the typed contract with prompt_format: :plan; prompt_format: :string explicitly selects compatibility serialization.

A fallback model independently negotiates the same capability, so a legacy primary and structured fallback (or the reverse) can share one prompt plan.

Summary

Types

completion()

completion_result()

prompt()

Callbacks

complete(t, keyword)

complete_plan(t, keyword)

Functions

complete(prompt, opts \\ [])

Completes a rendered string or typed prompt plan through the configured model adapter or function.

complete_once(prompt, opts \\ [])

Executes exactly one configured model without applying the legacy fallback.

provider_opts(opts, extra_keys \\ [])

Strips Spectre runtime context from an adapter keyword list.

runtime_opt_keys()

Returns the option keys the Spectre runtime injects into adapter calls.

Types

completion()

@type completion() :: String.t() | Spectre.Inference.Response.t() | map()

completion_result()

@type completion_result() :: {:ok, completion()} | {:error, term()}

prompt()

@type prompt() :: String.t() | Spectre.Prompt.Plan.t()

Callbacks

complete(t, keyword)

@callback complete(
  String.t(),
  keyword()
) ::
  {:ok, String.t() | Spectre.Inference.Response.t() | map()} | {:error, term()}

complete_plan(t, keyword)

(optional)

@callback complete_plan(
  Spectre.Prompt.Plan.t(),
  keyword()
) ::
  {:ok, String.t() | Spectre.Inference.Response.t() | map()} | {:error, term()}

Functions

complete(prompt, opts \\ [])

@spec complete(
  prompt(),
  keyword()
) :: completion_result()

Completes a rendered string or typed prompt plan through the configured model adapter or function.

Spectre.LLM.complete("Say hello", model: {MyApp.LLM, :complete, model: "small"})

complete_once(prompt, opts \\ [])

@spec complete_once(
  prompt(),
  keyword()
) :: completion_result()

Executes exactly one configured model without applying the legacy fallback.

Spectre.Inference uses this primitive so one boundary owns retry and profile fallback decisions.

provider_opts(opts, extra_keys \\ [])

@spec provider_opts(
  keyword(),
  [atom()]
) :: keyword()

Strips Spectre runtime context from an adapter keyword list.

Everything the runtime attaches for its own purposes — routing context, state, prompt assigns, model bookkeeping — is removed, along with any spectre_*-prefixed key, leaving only options intended for the provider. Adapters call this instead of maintaining their own allowlists:

def complete(prompt, opts) do
  provider_opts = Spectre.LLM.provider_opts(opts, [:test_pid])
  MyHTTP.post(url, body, provider_opts)
end

extra_keys removes host-specific keys on top of the runtime set.

runtime_opt_keys()

@spec runtime_opt_keys() :: [atom()]

Returns the option keys the Spectre runtime injects into adapter calls.