OTP Application for EMLX.
Allocates the application-default EMLX.CommandQueue (one per device:
:cpu and :gpu) at boot and stashes the NIF resource references in
:persistent_term. These are the workers used by EMLX.eval/1 and
EMLX.to_blob/1 for any process that has not bound its own queue via
EMLX.CommandQueue.with_queue/2.
Also allocates one runtime_call worker per device — see
runtime_call_worker/1's doc for why a Nx.runtime_call/4 callback's own
eager EMLX calls must never route back to the worker that is, right now,
blocked waiting for that very callback to return.
See clean-room-import/01-worker-thread-dispatch.md for the rationale
behind :persistent_term instead of a GenServer + Registry.
Idempotency
start/2 is safe to call more than once (e.g. from an umbrella where
:emlx is referenced by several apps). The first call to allocate a
worker for a given device wins; subsequent calls observe the existing
:persistent_term entry and skip. This matters because every
:persistent_term.put/2 triggers a global GC scan across every
process heap on the node — we do exactly two puts (CPU + GPU) over
the BEAM's lifetime.
GPU absence
On platforms where MLX cannot allocate a GPU stream (e.g. Linux without
Metal), EMLX.NIF.command_queue_new(:gpu) returns {:error, _} and
this module silently skips the GPU worker. Subsequent
EMLX.eval/1 calls on a GPU tensor will raise at use time with the
underlying :persistent_term ArgumentError.
CPU JIT compilation and SIGCHLD
Hitting ** (EMLX.NIFError) ... pclose() failed. on the CPU backend? See
the "CPU JIT compilation and SIGCHLD" section of the EMLX moduledoc —
it's a BEAM/MLX interaction this application deliberately does not paper
over for you.
Summary
Functions
Returns the application-default EMLX.CommandQueue NIF resource for
the given device.
Returns the EMLX.CommandQueue NIF resource dedicated to running
Nx.runtime_call/4 callbacks for the given device.
Functions
Returns the application-default EMLX.CommandQueue NIF resource for
the given device.
Raises ArgumentError if no worker has been allocated for the device
(e.g. asking for :gpu on a system without Metal).
Never call :persistent_term.put/2 on the underlying key
({EMLX, :default_worker, device}) — overwriting a :persistent_term
value triggers a node-wide GC.
Returns the EMLX.CommandQueue NIF resource dedicated to running
Nx.runtime_call/4 callbacks for the given device.
EMLX.handle_runtime_call/5 binds this worker (via
EMLX.CommandQueue.with_queue/2's same Process.put(:emlx_command_queue, _) mechanism) for the duration of every real callback invocation, so any
eager EMLX call the callback itself makes (e.g.
EMLX.Quantization.dequantize_callback/2 calling EMLX.dequantize/1)
gets worker-routed here instead of to the default (or bound) worker that
is, right now, blocked inside EMLXRuntimeCall::eval_cpu/eval_gpu
waiting for that very callback to finish. Reusing that same, still-busy
worker would either deadlock (its one dedicated OS thread cannot service
a new job while blocked — see emlx_worker.hpp) or, if the blocked wait
instead pumps that worker's own queue inline, crash (MLX's GPU eval()
is not reentrant on one OS thread — nesting a second mlx::core::eval
call while the first is still on the C++ stack corrupts Metal
command-buffer/completion-handler state). A second, otherwise-idle
worker sidesteps both failure modes: it is a distinct OS thread with its
own mlx::core::Stream, so its jobs never contend with (or nest inside)
the blocked worker's.
Raises ArgumentError if no worker has been allocated for the device
(e.g. asking for :gpu on a system without Metal) — same contract as
default_worker/1.