Limen.Sketch.HyperLogLog (limen v0.2.1)

Copy Markdown View Source

A lock-free HyperLogLog on :atomics.

Estimates the number of distinct keys added with a relative standard error of about 1.04 / sqrt(2^precision) (1.6% at the default precision of 12) in 2^precision bytes: registers are 8 bits wide and packed eight to a 64-bit atomic, updated with compare-and-swap.

Estimates use Otmar Ertl's improved estimator, which keeps that error across the whole range. The original estimator errs about twice as much around 2.5 × 2^precision keys, where it switches to counting empty registers.

Adding a key costs one hash and usually one atomic read; estimating scans every register, so estimates belong in background processes, not on the request path.

Summary

Functions

Estimates the number of distinct keys added to any of sketches.

Memory used by the registers, in bytes.

Creates a sketch with 2^precision registers.

Clears every register. Not atomic with respect to concurrent updates.

Types

t()

@type t() :: %Limen.Sketch.HyperLogLog{precision: 4..16, ref: :atomics.atomics_ref()}

Functions

add(hyper_log_log, key)

@spec add(t(), term()) :: :ok

Adds key.

cardinality(sketch)

@spec cardinality(t() | [t(), ...]) :: non_neg_integer()

Estimates the number of distinct keys added to any of sketches.

Passing several sketches of the same precision estimates their union.

memory(hyper_log_log)

@spec memory(t()) :: non_neg_integer()

Memory used by the registers, in bytes.

new(precision \\ 12)

@spec new(4..16) :: t()

Creates a sketch with 2^precision registers.

reset(hyper_log_log)

@spec reset(t()) :: :ok

Clears every register. Not atomic with respect to concurrent updates.