Candil.Engine (Candil v2.1.0)

Local llama.cpp engine definition and lifecycle management.

An engine represents a llama-server binary that serves a model over an OpenAI-compatible HTTP API. Multiple models can be configured to use the same engine definition, but only one model can be loaded per running server instance.

Fields

:alias — unique atom identifier for the engine
:binary_dir — directory where the llama-server binary lives or will be installed (default: "~/.apero/llm/bin")
:use_precompiled — if true, Apero will download the official precompiled binary from the llama.cpp GitHub releases (default: true)
:precompiled_version — :latest or a specific release tag such as "b4561" (default: :latest)
:host — host the server listens on (default: "127.0.0.1")
:port — base port; each running instance uses this port plus an offset (default: 8080)
:start_args — extra CLI arguments passed to llama-server at startup (e.g. ["--n-gpu-layers", "35"])
:launcher — module implementing Candil.Engine.Launcher to start the engine out-of-band (systemd, docker, another process). When set, Candil does NOT spawn llama-server itself and does NOT download the binary. Default: nil (default: Candil owns the binary lifecycle).

See Candil.Engine.Launcher for the custom-launcher contract.

Summary

Types

alias()

t()

version()

Functions

base_url(model_alias)

Returns the base URL for the engine serving model_alias, or nil if not running.

binary_dir(engine)

Returns the effective binary directory for an engine.

binary_exists?(engine)

Returns true if the engine binary exists on disk.

binary_path(engine)

Returns the full path to the llama-server binary for this engine.

healthy?(model_alias)

Returns true if the engine serving model_alias is running and responding to the /health endpoint.

registry()

Returns the Registry module used for engine registration.

start(engine, model)

Starts a llama-server process loaded with model.

stop(model_alias)

Stops the engine server running the given model alias.

Types

alias()

@type alias() :: atom()

t()

@type t() :: %Candil.Engine{
  alias: atom(),
  binary_dir: binary() | nil,
  checksum_sha256: binary() | nil,
  host: binary(),
  launcher: module() | nil,
  port: :inet.port_number(),
  precompiled_version: version(),
  start_args: [binary()],
  use_precompiled: boolean()
}

version()

@type version() :: :latest | binary()

Functions

base_url(model_alias)

@spec base_url(atom()) :: binary() | nil

Returns the base URL for the engine serving model_alias, or nil if not running.

binary_dir(engine)

@spec binary_dir(t()) :: binary()

Returns the effective binary directory for an engine.

Falls back to ~/.apero/llm/bin when binary_dir is nil.

binary_exists?(engine)

@spec binary_exists?(t()) :: boolean()

Returns true if the engine binary exists on disk.

binary_path(engine)

@spec binary_path(t()) :: binary()

Returns the full path to the llama-server binary for this engine.

healthy?(model_alias)

@spec healthy?(atom()) :: boolean()

Returns true if the engine serving model_alias is running and responding to the /health endpoint.

registry()

@spec registry() :: module()

Returns the Registry module used for engine registration.

Defaults to Candil.Registry. Can be configured via:

config :candil, :registry, MyApp.CustomRegistry

start(engine, model)

@spec start(t(), Candil.Model.t()) :: {:ok, pid()} | {:error, binary()}

Starts a llama-server process loaded with model.

If engine.use_precompiled is true and the binary does not exist, this function automatically downloads it before starting. If engine.launcher is set, the launcher is invoked instead and Candil never spawns the binary itself — see Candil.Engine.Launcher.

Registers the running server in Candil.Registry under the model alias. Returns {:ok, pid} or {:error, reason}.

stop(model_alias)

@spec stop(atom()) :: :ok | {:error, :not_running}

Stops the engine server running the given model alias.

Returns :ok or {:error, :not_running}.