erllama_model_llama (erllama v0.1.0)

View Source

Real-llama.cpp backend for erllama_model.

Owns a model_ref and a context_ref from erllama_nif. The gen_statem hands its decode/kv operations through this module; this module forwards to the NIF.

Config (passed through erllama_model:start_link/2): model_path :: file:name() | binary() (required) model_opts :: map() (forwarded to erllama_nif:load_model/2) context_opts :: map() (forwarded to erllama_nif:new_context/2)

Summary

Functions

apply_adapters/2

apply_chat_template/2

clear_sampler/1

configure_sampler/2

decode_one/2

detokenize/2

embed/2

init(Config)

kv_pack/2

kv_unpack/2

load_adapter/2

prefill/2

seq_rm_last/2

set_grammar/2

terminate/1

tokenize/2

unload_adapter/2