LlamaCppEx.Options (LlamaCppEx v0.8.41)

Copy Markdown View Source

Single owner for option policy shared across the public entry points.

Two things live here, both of which used to be duplicated per call site:

Scalar defaults

Context.tuning_option_keys/0 and LlamaCppEx.Sampler.option_keys/0 gave the option lists an owner, but the scalar defaults kept their own copies. There were six hand-rolled Keyword.get(opts, :timeout, ...) calls across three modules, split 60s/30s, and LlamaCppEx.stream_chat_completion/3 picked between them purely on the type of its first argument. The split itself is intentional — see blocking_timeout/0 and stream_timeout/0 — but it needs to be stated once.

Unknown-key rejection

Keyword.take/2 is the routing mechanism in this library: each consumer takes the keys it owns and ignores the rest. That makes an unknown key structurally indistinguishable from another module's key, so a typo is silently dropped — generate(model, prompt, temperature: 0.1) runs at the default temperature and n_paralell: 8 runs at 4. Routing therefore cannot validate; only the public entry point, which knows the complete key set, can.

This module deliberately has no dependencies on LlamaCppEx, LlamaCppEx.Server or LlamaCppEx.ModelManager, so any of them can use it without adding a cycle.

Summary

Functions

Default :timeout for a call that blocks until generation completes.

Default :timeout for a streaming call, bounding the wait per chunk.

Reads :timeout from opts, defaulting by call shape.

Raises ArgumentError unless every key in opts appears in known.

Functions

blocking_timeout()

@spec blocking_timeout() :: pos_integer()

Default :timeout for a call that blocks until generation completes.

stream_timeout()

@spec stream_timeout() :: pos_integer()

Default :timeout for a streaming call, bounding the wait per chunk.

timeout(opts, atom)

@spec timeout(
  keyword(),
  :blocking | :stream
) :: timeout()

Reads :timeout from opts, defaulting by call shape.

mode is :blocking or :stream. :infinity is accepted.

validate!(opts, known, label)

@spec validate!(keyword(), [atom()], String.t()) :: keyword()

Raises ArgumentError unless every key in opts appears in known.

label names the entry point in the error message. Returns opts unchanged so it can sit in a pipeline. A near-miss key gets a "did you mean" hint, since the motivating failures are typos (temperature for temp, n_paralell for n_parallel) rather than wholly invented options.