sluice/pool
A pooled sink. It runs each event in a separate worker process, with a limit on the quantity of parallel workers. Demand follows the workers: each worker asks for one more event through the subscription when it completes. Thus the backpressure of the pipeline follows the true work speed.
The limit applies for each subscription of the pool. A failure in a worker does not stop the pool. The pool writes a warning to the log and continues.
Use pool.sink in the place of sink.new, and set the demand mode of
the subscription to Manual. The pool makes its own asks. Do not
change the subscription hooks of a pool sink.
Types
Values
pub fn sink(
concurrency concurrency: Int,
run run: fn(event) -> Nil,
) -> sink.Builder(State, event)Define a pooled sink: a maximum of concurrency workers run at the
same time, and each worker runs run for one event. The result is a
normal sink builder: give it a name or start it directly, and
subscribe it with the Manual demand mode.
concurrency: The maximum quantity of parallel workers for each subscription.run: The function that one worker runs for one event.