sbroker_codel_queue (sbroker v1.1.1)
Implements a head or tail drop queue with queue management based on CoDel (Controlling Queue Delay).
sbroker_codel_queue
can be used as a sbroker_queue
in a sbroker
or sregulator
. It will provide a FIFO or LIFO queue that drops request in the queue using CoDel when the minimum size is exceeded, and drops the head or tail request from the queue when a maximum size is exceeded. Its argument, spec()
, is of the form: #{out => Out :: out | out_r, % default: out
target => Target :: non_neg_integer(), % default: 100
interval => Interval :: pos_integer(), % default: 1000
drop => Drop :: drop | drop_r, % default: drop_r
min => Min :: non_neg_integer(), % default: 0
max => Max :: non_neg_integer() | infinity} % default: infinity
Out
is either out
for a FIFO queue (the default) or out_r
for a LIFO queue. Target
is the target queue sojourn time in milliseconds (defaults to 100
). Interval
is in the initial interval in milliseconds (defaults to 1000
) between drops when the queue is above the minimum size Min
(defaults to 0
). Drop
is either drop_r
for tail drop (the default) where the last request is droppped, or drop
for head drop, where the first request is dropped. Dropping occurs when queue is above the maximum size Max
(defaults to infinity
).
Initial parameters are recommended to be between the 95th and 99th percentile round trip time for the interval and the target between 5% and 10% of the interval. The round trip should be that between the actual initiator of the request (e.g. a remote client) and the queue. For example, the reference suggests an interval of 100ms and a target of 5ms when queuing TCP packets in a kernel's buffer. A request and response might be a few more round trips at the packet level even if using a single :gen_tcp.send/2
.
A person perceives a response time of 100
milliseconds or less as instantaneous, and feels engaged with a system if response times are 1000
milliseconds or less. Therefore it is desirable for a system to respond within 1000
milliseconds as a worst case (upper percentile response time) and ideally to respond within 100
milliseconds (target response time). These also match with the suggested ratios for CoDel and so the default target is 100
milliseconds and the default is interval 1000
milliseconds.
This implementation differs from the reference as enqueue and other functions can detect a slow queue and drop items. However once a slow item has been detected only handle_out/2
can detect the queue becoming fast again with the caveat that with out_r
this can only be detected if the queue reaches the minimum size Min
or less, and not a fast sojourn time. This means that it is possible to drop items without calling handle_out/2
but it is not possible to stop dropping unless a handle_out/2
dequeues an item to take the queue to or below the minimum size Min
or out
is used and a dequeued request is below target sojourn time. Therefore if handle_out/2
is not called for an extended period the queue will converge to dropping all items above the target sojourn time if a single item has a sojourn time above target. Whereas with the reference implementation no items would be dropped.
Min
above 0
can leave Min
items in the queue for an extended period of time as requests are only dropped when the queue size is above Min
. This may be undesirable for client requests because the request could wait in the queue indefinitely if there are not enough requests to take the queue above Min
. However it might be desired for database connections where it is ideal for a small number of connections to be waiting to handle a client request.
Link to this section Summary
Link to this section Types
spec/0
Specs
spec() :: #{out => Out :: out | out_r, target => Target :: non_neg_integer(), interval => Interval :: pos_integer(), drop => Drop :: drop | drop_r, min => Min :: non_neg_integer(), max => Max :: non_neg_integer() | infinity}.