Candil.Engine (Candil v1.0.0)

Copy Markdown View Source

Local llama.cpp engine definition and lifecycle management.

An engine represents a llama-server binary that serves a model over an OpenAI-compatible HTTP API. Multiple models can be configured to use the same engine definition, but only one model can be loaded per running server instance.

Fields

  • :alias — unique atom identifier for the engine
  • :binary_dir — directory where the llama-server binary lives or will be installed (default: "~/.apero/llm/bin")
  • :use_precompiled — if true, Apero will download the official precompiled binary from the llama.cpp GitHub releases (default: true)
  • :precompiled_version — :latest or a specific release tag such as "b4561" (default: :latest)
  • :host — host the server listens on (default: "127.0.0.1")
  • :port — base port; each running instance uses this port plus an offset (default: 8080)
  • :start_args — extra CLI arguments passed to llama-server at startup (e.g. ["--n-gpu-layers", "35"])

Summary

Types

t()
version()

Functions

base_url(model_alias)

Returns the base URL for the engine serving model_alias, or nil if not running.

binary_dir(engine)

Returns the effective binary directory for an engine.

binary_exists?(engine)

Returns true if the engine binary exists on disk.

binary_path(engine)

Returns the full path to the llama-server binary for this engine.

healthy?(model_alias)

Returns true if the engine serving model_alias is running and responding to the /health endpoint.

start(engine, model)

Starts a llama-server process loaded with model.

stop(model_alias)

Stops the engine server running the given model alias.

Types

t()

@type t() :: %Candil.Engine{
  alias: atom(),
  binary_dir: binary() | nil,
  host: binary(),
  port: :inet.port_number(),
  precompiled_version: version(),
  start_args: [binary()],
  use_precompiled: boolean()
}

version()

@type version() :: :latest | binary()

Functions

base_url(model_alias)

@spec base_url(atom()) :: binary() | nil

Returns the base URL for the engine serving model_alias, or nil if not running.

binary_dir(engine)

@spec binary_dir(t()) :: binary()

Returns the effective binary directory for an engine.

Falls back to ~/.apero/llm/bin when binary_dir is nil.

binary_exists?(engine)

@spec binary_exists?(t()) :: boolean()

Returns true if the engine binary exists on disk.

binary_path(engine)

@spec binary_path(t()) :: binary()

Returns the full path to the llama-server binary for this engine.

healthy?(model_alias)

@spec healthy?(atom()) :: boolean()

Returns true if the engine serving model_alias is running and responding to the /health endpoint.

start(engine, model)

@spec start(t(), Candil.Model.t()) :: {:ok, pid()} | {:error, binary()}

Starts a llama-server process loaded with model.

If engine.use_precompiled is true and the binary does not exist, this function automatically downloads it before starting.

Registers the running server in Candil.Registry under the model alias. Returns {:ok, pid} or {:error, reason}.

stop(model_alias)

@spec stop(atom()) :: :ok | {:error, :not_running}

Stops the engine server running the given model alias.

Returns :ok or {:error, :not_running}.