livery_concurrency (livery v0.2.0)
View SourceConcurrency-limit / load-shedding middleware (admission control).
Caps the number of in-flight requests. Over the limit it sheds load
immediately with 503 Service Unavailable and never calls the handler;
at or under the limit the request proceeds and the slot is released when
the handler returns.
Build the stack entry with the limiter/1,2 factory, which creates the
shared counter once:
Stack = [
{livery_concurrency, livery_concurrency:limiter(1000)}
%% ... handler runs only while < 1000 requests are in flight
].The counter is a lock-free atomics cell shared across the request
processes (no extra process). A global limiter is one factory call in
the service stack; per-route limiters are independent factory calls.
Scope: a slot is held from admission until the handler RETURNS its response. Body streaming happens after that (outside the middleware stack), so the slot does not cover the duration of a streamed/SSE body. The limit is approximate under a burst (a request that increments past the limit decrements again), which is acceptable for load-shedding.
Summary
Types
-type state() :: #{ref := atomics:atomics_ref(), limit := non_neg_integer(), status := 100..599, body := iodata(), retry_after := non_neg_integer() | binary() | undefined}.
Functions
-spec call(livery_req:req(), livery_middleware:next(), state()) -> livery_resp:resp().
Admit the request if under the limit, otherwise shed with 503.
-spec limiter(non_neg_integer()) -> state().
Build a limiter stack State capping in-flight requests at Limit.
-spec limiter(non_neg_integer(), map()) -> state().
limiter/1 with options.
status (default 503), body (default <<"service unavailable">>),
and retry_after (seconds as an integer, a literal binary, or
undefined) shape the shed response.