LlamaCppEx.ModelManager.Entry (LlamaCppEx v0.8.23)

A single resident-model record held in the LlamaCppEx.ModelManager ETS table.

Entries carry raw NIF resources (:model) and backing process pids (:server_pid). to_public/1 produces a sanitized, reference-free view for LlamaCppEx.ModelManager.list/0 and info/1 — handing out the raw %Model{} would keep the model alive past unload/1 and defeat GC-based reclamation.

Summary

Types

capability()

mode()

source()

status()

t()

Functions

to_public(e)

Returns a sanitized map view of an entry — no %Model{} refs or pids.

Types

capability()

@type capability() :: :generate | :chat | :embed

mode()

@type mode() :: :server | :direct

source()

@type source() :: {:path, String.t()} | {:hub, String.t(), String.t()}

status()

@type status() :: :loading | :ready | :unloading | :error

t()

@type t() :: %LlamaCppEx.ModelManager.Entry{
  byte_size: non_neg_integer(),
  capabilities: [capability()],
  error: term(),
  est_bytes: non_neg_integer(),
  id: term(),
  last_used: integer(),
  loaded_at: integer() | nil,
  mode: mode(),
  model: LlamaCppEx.Model.t() | nil,
  monitor_ref: reference() | nil,
  n_gpu_layers: integer(),
  placement: %{
    ram: non_neg_integer(),
    vram: %{required(non_neg_integer()) => non_neg_integer()}
  },
  server_pid: pid() | nil,
  source: source() | nil,
  status: status()
}

Functions

to_public(e)

@spec to_public(t()) :: map()

Returns a sanitized map view of an entry — no %Model{} refs or pids.