API Reference erllama v#0.10.1

Public API of erllama: load llama.cpp models as supervised OTP processes, run completions, stream tokens, and reuse the KV cache.

Public API of the KV cache: tier setup, inspection, eviction and counters.

Middleware around the erllama API calls, in the style of hackney's middleware: a middleware is a plain fun that receives the request and a Next fun, and returns the response. It can observe, rewrite, short-circuit or wrap a call. No behaviour, no registry, no dependencies.

Behaviour describing the operations the erllama_model gen_statem needs from a backing inference engine.

Behaviour for proactive model eviction under memory pressure.

Deterministic test backend: no NIF, no GGUF, no hardware.

Behaviour and helpers for memory-pressure samplers used by erllama_scheduler. A sampler is a stateless module that returns the current {Used, Total} byte tuple for the resource it tracks (system RAM, GPU VRAM, or a custom source).

Memory-pressure-driven cache eviction.