Opt-in supervisor for the multi-model manager.
Starts, in order, the Registry and DynamicSupervisor that server-backed
models need, then LlamaCppEx.ModelManager. Add it to your application's
supervision tree:
children = [
{LlamaCppEx.ModelSupervisor,
memory_budget: :auto,
models: [
{"chat", {:hub, "Qwen/Qwen3-0.6B-GGUF", "Qwen3-0.6B-Q8_0.gguf"}, n_gpu_layers: -1},
{"embed", {:path, "/models/nomic-embed.gguf"}, capabilities: [:embed]}
]}
]For quick scripts or IEx, start it directly:
{:ok, _sup} = LlamaCppEx.ModelSupervisor.start_link([])
{:ok, "chat"} = LlamaCppEx.ModelManager.load("chat", {:path, "model.gguf"})Options
:memory_budget- Forwarded toLlamaCppEx.ModelManager(:infinity,:auto, or a byte limit).:models- Models to auto-load after start (loaded by the manager so the supervisor itself does not block on downloads).:name- Names this supervisor. Defaults toLlamaCppEx.ModelSupervisor. It does not rename the manager:LlamaCppEx.ModelManageris a node-wide singleton registered under its module name (the client API targets it there), so only oneModelSupervisorshould run per node.
Summary
Functions
Returns a specification to start this module under a supervisor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec start_link(keyword()) :: Supervisor.on_start()