Imp.Clients.MLXLMDeployment (Imp v0.5.0)

Copy Markdown View Source

Supervised local serving for a verified fused MLXLMTrainer artifact.

start/2 revalidates the complete fused tree, launches the pinned mlx_lm.server command recorded by the training job, and succeeds only when /v1/models advertises the exact fused path. The returned ReqLLM is therefore bound to verified trained bytes rather than a model-name substitution.

Every launch receives a new empty, deployment-owned Hugging Face, Transformers, and XDG cache environment. The server still receives the verified local artifact path explicitly; ambient model catalogs cannot add a second advertised identity. The owned cache is removed after startup failure, explicit stop, server exit, or application shutdown.

Deployments are shared per fused artifact inside the Imp application and can be stopped explicitly with stop/1. Application shutdown also terminates the supervised external process group.

Summary

Types

t()

Functions

lookup(training_job)

Returns the live deployment for a job, if one exists.

start(job, opts \\ [])

Starts or reuses the supervised server for a completed fused MLX job.

stop(arg1)

Stops the supervised server for a deployment or completed MLX job.

Types

t()

@type t() :: %Imp.Clients.MLXLMDeployment{
  artifact_path: Path.t(),
  artifact_sha256: String.t(),
  base_url: String.t(),
  lm: Imp.Clients.ReqLLM.t(),
  pid: pid()
}

Functions

lookup(training_job)

@spec lookup(Imp.Clients.TrainingJob.t()) :: {:ok, t()} | :error

Returns the live deployment for a job, if one exists.

start(job, opts \\ [])

@spec start(Imp.Clients.TrainingJob.t(), keyword()) :: {:ok, t()} | {:error, term()}

Starts or reuses the supervised server for a completed fused MLX job.

stop(arg1)

@spec stop(t() | Imp.Clients.TrainingJob.t()) :: :ok | {:error, term()}

Stops the supervised server for a deployment or completed MLX job.