StatBuffer v1.1.0 StatBuffer behaviour View Source
Defines a stat buffer.
A stat buffer is an efficient way to maintain a local incrementable count with a given key that can later be flushed to persistent storage. In fast moving systems, this provides a scalable way keep track of counts without putting heavy loads on a database.
Creating a buffer is as easy as:
defmodule Buffer do
use StatBuffer
end
Once we have defined our buffer module, we must then implement the handle_flush/2
callback that allows us to perform an operation with a rovided key and counter.
This could mean something like updating a counter in a database.
defmodule Buffer do
use StatBuffer
def handle_flush(key, counter) do
# write to the database...
# handle_flush MUST return an :ok atom
:ok
end
end
We must then add our buffer to our supervision tree.
children = [
Buffer
]
Each flush operation is handled with its own supervised Task
process. By
default, a failed flush operation will retry about 3 times within 3 seconds.
Usage
With our buffer started, we can now increment key counters. A key can be any valid term.
Buffer.increment("mykey") # increments by 1
Buffer.increment("mykey", 10) # increments by 10
Key counts are maintained in an ETS table. All keys are scoped to the given buffer module - so multiple buffers using the same keys will not cause issues.
With the default buffer we setup above, the "mykey" counter will be flushed after 5 seconds. Assuming no new operations occur with our buffer, the process will be placed into hibernation after 10 seconds. All of this is configurable through the options below.
Options
A stat buffer comes with a few configurable options. We can pass any of these options along with the use macro.
use StatBuffer, interval: 60_000, jitter: 20_000
:interval
- the time in milliseconds between the first increment for a given key and its next flush callback being invoked. Defaults to5_000
.:jitter
- a max time in milliseconds that will be added tointerval
to ensure some randomness in each flush invocation. The time added would be randomly selected between 0 andjitter
. Defaults to0
.:timeout
- the time in milliseconds between the last operation on a a buffer, and the process being hibernated. Defaults to10_000
.:backoff
- the time in milliseconds between ahandle_flush/2
callback failing, and the next attempt occuring. Defaults to1_000
.
Link to this section Summary
Callbacks
This callback has been deprecated - use increment/2
instead.
Returns the current state of a key from the buffer.
Asynchronously flushes a given key from the buffer.
Callback for flushing a key for the buffer.
Increments a given key in the buffer by the provided count.
Starts the buffer process.
Link to this section Types
option()
View Source
option() ::
{:interval, non_neg_integer()}
| {:jitter, non_neg_integer()}
| {:timeout, non_neg_integer()}
| {:backoff, non_neg_integer()}
option() :: {:interval, non_neg_integer()} | {:jitter, non_neg_integer()} | {:timeout, non_neg_integer()} | {:backoff, non_neg_integer()}
options()
View Source
options() :: [option()]
options() :: [option()]
t()
View Source
t() :: module()
t() :: module()
Link to this section Callbacks
async_increment(key, count) View Source
This callback has been deprecated - use increment/2
instead.
count(key) View Source
Returns the current state of a key from the buffer.
flush(key)
View Source
flush(key :: any()) :: :ok
flush(key :: any()) :: :ok
Asynchronously flushes a given key from the buffer.
handle_flush(key, counter) View Source
Callback for flushing a key for the buffer.
When a buffer key hits its set time interval, this function will be called and provided with the key as well its current counter.
This function is called within its own Task and is supervised. If the
callback does not return :ok
- the task will fail and attempt a retry
with configurable backoff.
increment(key, count) View Source
Increments a given key in the buffer by the provided count.
Each key is scoped to the buffer module. So duplicate keys across different buffer modules will not cause issues.
start_link(options)
View Source
start_link(options()) :: GenServer.on_start()
start_link(options()) :: GenServer.on_start()
Starts the buffer process.
Options
The options available are the same provided in the "Options" section.