@spec generate(t(), String.t(), keyword()) :: {:ok, String.t()} | {:error, term()}

Synchronously generates text. Equivalent to running stream/3 and joining the pieces into a single binary.

Accepts the same options as stream/3.

@spec init(
  LlamaCppEx.Model.t(),
  keyword()
) :: {:ok, t()} | {:error, term()}

Initializes an MTP speculative session: builds the target context, the MTP draft context (ctx_type: :mtp), and the underlying common_speculative state.

Options

• :draft_model - A separate LlamaCppEx.Model holding the MTP head, for checkpoints that ship it as a sidecar GGUF rather than inside the target file (Qwen 3.8 is the current example: Qwen3.8-27B-Q4_K_M.gguf plus mtp-Qwen3.8-27B-Q4_0.gguf). It must be loaded with load_mtp: true. Defaults to nil, meaning the head is expected inside the target model and the draft context is built against it.
• :n_draft - Max draft tokens generated per iteration. Defaults to 3. Larger values mean fewer model forward passes but lower per-iteration acceptance; 2–4 is the sweet spot in practice.
• :n_ctx - Context size for both contexts. Defaults to 2048.
• Any LlamaCppEx.Context option (e.g. :n_threads, :flash_attn, :type_k/:type_v, :offload_kqv). The same options are applied to both the target and draft contexts.

Returns {:ok, %MTP{}} or {:error, reason}.

@spec print_stats(t()) :: :ok

Writes upstream's own speculative stats summary to stdout (via llama.cpp logging). Useful when cross-checking acceptance rates against the upstream llama-server benchmark output.

@spec stats(t()) :: map()

Returns the current MTP statistics snapshot (lock-free read of atomic counters). Safe to call at any time — including from another process while a stream is in flight.

Returns a map with keys:

• :iters - speculative loop iterations completed
• :drafts_generated - draft tokens proposed by the MTP head
• :drafts_accepted - draft tokens accepted by the target model
• :acceptance_rate - drafts_accepted / drafts_generated (0.0–1.0)
• :tokens_emitted - tokens streamed back to the caller
• :tokens_per_sec - throughput over the active generation window
• :timing_us - %{draft: μs, verify: μs, sample: μs, ckpt: μs, other: μs, total: μs}. :ckpt is the recurrent-state save/restore that only hybrid models pay and is zero elsewhere; on Qwen 3.8 it is large enough to decide whether speculation helps at all. :other is whatever falls outside the named buckets, dominated on Metal by GPU-sync waits from the previous iteration's async verify decode.
• :n_draft - max draft length configured at init

Counters are cumulative across all stream/3 / generate/3 calls on this MTP value.

@spec stream(t(), String.t(), keyword()) :: Enumerable.t()

Returns a lazy stream of generated text pieces.

Options

• :max_tokens - Maximum tokens to generate (default 256).
• :emit_stats_every - When > 0, also emits {:stats, snapshot_map} events every Nth token via the underlying message stream. Note: these events are filtered out of this String.t() stream — to consume them use stream_events/3 instead. Default 0 (off).
• :timeout - Receive timeout in milliseconds (default 60_000).
• Any sampler option from LlamaCppEx.Sampler.create/2 (:temp, :top_k, :top_p, :min_p, :seed, :penalty_*, :grammar, etc.).

Each emitted element is the text piece for one accepted token. The stream ends on end-of-generation, max-tokens, or error.

@spec stream_events(t(), String.t(), keyword()) :: Enumerable.t()

Like stream/3, but yields the raw event tuples emitted by the NIF:

• {:token, token_id, text_piece} - one accepted token
• {:stats, snapshot_map} - periodic stats (only when :emit_stats_every > 0)
• {:done, final_stats_map} - generation completed normally
• {:eog, nil} - model emitted an end-of-generation token

The stream halts after :done / :eog / :error. The final stats map is available via stats/1 on the MTP struct even after the stream ends.