Histogrex (Histogrex v0.0.5)

High Dynamic Range (HDR) Histogram allows the recording of values across a configurable range at a configurable precision.

Storage requirement is fixed (and depends on how the histogram is configured). All functions are fully executed within the calling process (not serialized through a single process) as the data is stored within write-optimized ETS table. Each recording consists of as single call to :ets.update_counter.

Read operations consiste of a single :ets.lookup and are subsequently processed on that copy of the data (again, within the calling process).

The fist step involves creating a registry:

defmodule MyApp.Stats do

use Histogrex

histogrex :load_user, min: 1, max: 10_000_000, precision: 3
histogrex :db_save_settings, min: 1, max: 10_000, precision: 2
...

end

And then adding this module as a worker to your application's supervisor tree:

worker(MyApp.Stats, [])

You can then record values and make queries:

alias MyApp.Stats

Stats.record!(:load_user, 233)
Stats.record!(:db_save_settings, 84)

Stats.mean(:load_user)
Stats.max(:db_save_settings)
Stats.total_count(:db_save_settings)
Stats.value_at_quantile(:load_user, 99.9)

Link to this section Summary

Functions

Deletes the histogram. The histogram can no longer be used.

Deletes the histogram. Since this histogram was dynamically created through a template, you can safely continue using it.

registers the histogram

Gets the approximate maximum value recorded. Works both with Iterator and Histogram

Returns the mean value

Gets the approximate minimum value recorded. Works both with Iterator and Histogram

Creates a new histogrex object. Note that this only creates the configuration structure. It does not create the underlying ets table/entries. There should be no need to call this direction. Use the histogrex macro instead.

Records the value n times where n defaults to 1. Return :ok on success or {:error, message} on failure. A larger than the 'max' specified when the histogram was created will cause an error. This is usually not called directly, but rather through the record/3 of your custom registry

Records the value n times where n defaults to 1. Uses the template to create the histogram if it doesn't already exist. Return :ok on success or {:error, message} on failure. A larger than the 'max' specified when the histogram was created will cause an error. This is usually not called directly, but rather through the record/3 of your custom registry

Same as record/3 but raises on error

Same as record_template/4 but raises on error

Reduce all of a registry's histograms

Resets the histogram to 0 values. Note that the histogram is a fixed-size, so calling this won't free any memory. It is useful for testing.

Get the total number of recorded values. This is O(1)

Gets the value at the requested quantile. The quantile must be greater than 0 and less than or equal to 100. It can be a float.

Link to this section Types

@type t() :: %Histogrex{
  bucket_count: pos_integer(),
  counts_length: pos_integer(),
  name: atom() | binary(),
  registrar: module(),
  sub_bucket_count: non_neg_integer(),
  sub_bucket_half_count: non_neg_integer(),
  sub_bucket_half_count_magnitude: non_neg_integer(),
  sub_bucket_mask: non_neg_integer(),
  template: nil | tuple(),
  unit_magnitude: non_neg_integer()
}

Link to this section Functions

@spec delete(t()) :: :ok

Deletes the histogram. The histogram can no longer be used.

Link to this function

delete(h, metric)

@spec delete(t(), atom() | binary()) :: :ok

Deletes the histogram. Since this histogram was dynamically created through a template, you can safely continue using it.

Link to this macro

histogrex(name, opts)

(macro)

registers the histogram

A min, max and precision must be supplied. min and max represent the minimal and maximal expected values. For example, if you're looking to measure how long a database call took, you could specify: min: 1, max: 10_000 and provide time in millisecond, thus allowing you to capture values from 1ms to 10sec.

min must be greater than 0. max must be greater than min. precision must be between 1 and 5 (inclusive).

examples

Examples

histogrex :user_list, min: 1, max: 10000000, precision: 3
Link to this function

lowest_equivalent_value(h, value)

Link to this function

lowest_equivalent_value(h, bucket_index, sub_bucket_index)

@spec max(t() | Histogrex.Iterator.t()) :: non_neg_integer()

Gets the approximate maximum value recorded. Works both with Iterator and Histogram

@spec mean(t() | Histogrex.Iterator.t()) :: float()

Returns the mean value

Link to this function

mean(h, metric)

@spec min(t() | Histogrex.Iterator.t()) :: non_neg_integer()

Gets the approximate minimum value recorded. Works both with Iterator and Histogram

Link to this function

new(name, registrar, min, max, precision \\ 3, template \\ false)

@spec new(binary() | atom(), module(), pos_integer(), pos_integer(), 1..5, boolean()) ::
  t()

Creates a new histogrex object. Note that this only creates the configuration structure. It does not create the underlying ets table/entries. There should be no need to call this direction. Use the histogrex macro instead.

Link to this function

record(err, value, n)

@spec record(t() | {:error, any()}, pos_integer(), pos_integer()) ::
  :ok | {:error, any()}

Records the value n times where n defaults to 1. Return :ok on success or {:error, message} on failure. A larger than the 'max' specified when the histogram was created will cause an error. This is usually not called directly, but rather through the record/3 of your custom registry

Link to this function

record(err, metric, value, n)

@spec record(t() | {:error, any()}, atom() | binary(), pos_integer(), pos_integer()) ::
  :ok | {:error, any()}

Records the value n times where n defaults to 1. Uses the template to create the histogram if it doesn't already exist. Return :ok on success or {:error, message} on failure. A larger than the 'max' specified when the histogram was created will cause an error. This is usually not called directly, but rather through the record/3 of your custom registry

Link to this function

record!(h, value, n)

@spec record!(t(), pos_integer(), pos_integer()) :: :ok | no_return()

Same as record/3 but raises on error

Link to this function

record!(template, metric, value, n)

@spec record!(t(), atom() | binary(), pos_integer(), pos_integer()) ::
  :ok | no_return()

Same as record_template/4 but raises on error

Link to this function

reduce(module, acc, fun)

@spec reduce(module(), any(), (Histogrex.Iterator.t(), any() -> any())) :: any()

Reduce all of a registry's histograms

@spec reset(t() | Histogrex.Iterator.t()) :: :ok

Resets the histogram to 0 values. Note that the histogram is a fixed-size, so calling this won't free any memory. It is useful for testing.

Link to this function

reset(h, metric)

@spec reset(t(), atom() | binary()) :: :ok
Link to this macro

template(name, opts)

(macro)
@spec total_count(t() | Histogrex.Iterator.t()) :: non_neg_integer()

Get the total number of recorded values. This is O(1)

Link to this function

total_count(h, metric)

@spec total_count(t(), atom() | binary()) :: non_neg_integer()
Link to this function

value_at_quantile(h, q)

@spec value_at_quantile(t() | Histogrex.Iterator.t(), float()) :: float()

Gets the value at the requested quantile. The quantile must be greater than 0 and less than or equal to 100. It can be a float.

Gets the value at the requested quantile using the given iterator. When doing multiple calculations, it is slightly more efficent to first recreate and then re-use an iterator (plus the values will consistently be calculated based on the same data). Iterators are automatically reset before each call.

Gets the value at the requested quantile for the templated histogram

Link to this function

value_at_quantile(h, metric, q)

@spec value_at_quantile(t(), atom(), float()) :: float()