Registry of in-flight LLM generations, keyed by a cancel_ref/0.
Callers can request cancellation of an ongoing generation by passing
the reference to cancel/1. The current generation checks the
registry at chunk boundaries; if the reference is cancelled it
returns {:halt, :cancelled} from the streaming parser.
This is intentionally lightweight — a Registry with
partitions: 1 for low-overhead inserts, and ETS-backed counters
for live counts.
Summary
Functions
Cancel an in-flight generation. Returns :ok if cancelled,
:not_found if the ref is unknown (already done).
Has the ref been cancelled? Returns true or false. Returns
false if the ref is unknown.
Returns a specification to start this module under a supervisor.
Number of in-flight generations across all owners.
Mark a generation as done. Idempotent.
Generate a new cancellation reference.
Mark a generation as in-flight under the given ref.
Functions
@spec cancel(reference()) :: :ok | :not_found
Cancel an in-flight generation. Returns :ok if cancelled,
:not_found if the ref is unknown (already done).
Has the ref been cancelled? Returns true or false. Returns
false if the ref is unknown.
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec count() :: non_neg_integer()
Number of in-flight generations across all owners.
@spec done(reference()) :: :ok
Mark a generation as done. Idempotent.
@spec new_ref() :: reference()
Generate a new cancellation reference.
Mark a generation as in-flight under the given ref.
Stores owner_pid so that cancellations only fire from the same
process that started the generation (prevents accidental
cancellation from a different request).