ExDataSketch.Sketch behaviour (ExDataSketch v0.10.1)

Copy Markdown View Source

Behaviour implemented by every concrete sketch module.

This is the unified contract referenced by ExDataSketch.sketches/0 and by the top-level facade in ExDataSketch (new/2, update/2, merge/2, and so on). Elixir's @behaviour has no runtime dispatch machinery of its own -- implemented?/1 below is the only place this module actually calls function_exported?/3, to verify a candidate module's conformance (used by the sketches/0 registry test). Everywhere else -- the facade, and ExDataSketch.GenStage.SketchConsumer's direct sketch_module.merge/2/ update_many/2 calls -- calls the concrete module's function directly, relying on the behaviour purely as a compile-time-checked contract that the call will succeed, not as something with its own dispatch behavior. ExDataSketch.FilterChain additionally implements this behaviour itself (@behaviour ExDataSketch.Sketch) so the facade and ExDataSketch.Server can dispatch on it the same way they do for any other sketch module.

Which functions are required

Every concrete sketch module (ExDataSketch.HLL, ExDataSketch.Bloom, and so on) implements serialize/1, deserialize/1, size_bytes/1, and capabilities/0 unconditionally. new/1, update/2, update_many/2, and merge/2 are declared @optional_callbacks because not every family supports every operation with that exact shape:

A module's capabilities/0 is the source of truth for which of these optional operations it actually supports at runtime; @optional_callbacks only relaxes the compile-time contract so dialyzer does not require every module to implement every function.

capabilities/0 returns a MapSet, not a boolean map

Seven filter modules (ExDataSketch.Bloom, Cuckoo, Quotient, CQF, XorFilter, IBLT, FilterChain) already ship capabilities/0 returning a MapSet.t(atom()) of supported operation names (:put, :merge, :member?, :delete, and so on), consumed today by ExDataSketch.FilterChain's stage-composition checks. This behaviour standardizes on that existing, tested shape for all 16 families rather than introducing a second, incompatible capabilities/0 return type.

Adding @behaviour ExDataSketch.Sketch to a module

defmodule MyApp.Sketch do
  @behaviour ExDataSketch.Sketch

  defstruct [:state]

  @impl true
  def new(_opts \\ []), do: %__MODULE__{state: <<>>}

  @impl true
  def update(sketch, _item), do: sketch

  @impl true
  def serialize(%__MODULE__{state: state}), do: state

  @impl true
  def deserialize(binary), do: {:ok, %__MODULE__{state: binary}}

  @impl true
  def size_bytes(%__MODULE__{state: state}), do: byte_size(state)

  @impl true
  def capabilities, do: MapSet.new([:new, :update, :serialize, :deserialize])
end

Examples

iex> ExDataSketch.Sketch.implemented?(ExDataSketch.Bloom)
true

iex> ExDataSketch.Sketch.implemented?(String)
false

Summary

Types

The set of operation names a module reports as supported, as returned by capabilities/0. Existing members in production code include :new, :put, :put_many, :member?, :merge, :merge_many, :count, :serialize, :deserialize, :compatible_with?, :delete, :estimate_count, :subtract, :list_entries, and :add_stage.

A struct produced by a module implementing this behaviour.

Callbacks

Returns the set of operation names this module supports.

Deserializes a binary produced by serialize/1 back into a sketch.

Merges two sketches of the same family and compatible parameters.

Creates a new sketch with the given options.

Serializes a sketch to its canonical binary representation.

Returns the size, in bytes, of the sketch's serialized state.

Updates a sketch with a single item, returning the updated sketch.

Updates a sketch with every item in an enumerable, returning the updated sketch.

Functions

Returns true if module implements every non-optional callback of this behaviour (serialize/1, deserialize/1, size_bytes/1, capabilities/0).

Types

capabilities()

@type capabilities() :: MapSet.t(atom())

The set of operation names a module reports as supported, as returned by capabilities/0. Existing members in production code include :new, :put, :put_many, :member?, :merge, :merge_many, :count, :serialize, :deserialize, :compatible_with?, :delete, :estimate_count, :subtract, :list_entries, and :add_stage.

sketch()

@type sketch() :: struct()

A struct produced by a module implementing this behaviour.

Callbacks

capabilities()

@callback capabilities() :: capabilities()

Returns the set of operation names this module supports.

See capabilities/0 for the vocabulary of operation names in use.

deserialize(binary)

@callback deserialize(binary()) :: {:ok, sketch()} | {:error, Exception.t()}

Deserializes a binary produced by serialize/1 back into a sketch.

Returns {:error, exception} for malformed or corrupted input rather than raising, so callers can handle untrusted or persisted binaries safely.

merge(sketch, sketch)

(optional)
@callback merge(sketch(), sketch()) :: sketch() | {:error, Exception.t()}

Merges two sketches of the same family and compatible parameters.

Returns the merged sketch, or {:error, exception} when the two sketches have incompatible parameters (for example, different HLL precision).

Optional: not implemented by ExDataSketch.Cuckoo (bucket state is not associatively mergeable) or ExDataSketch.XorFilter (immutable).

new(opts)

(optional)
@callback new(opts :: keyword()) :: sketch()

Creates a new sketch with the given options.

Optional: ExDataSketch.XorFilter has no new/1 (its constructor is build/2, which requires the full item set upfront) and ExDataSketch.FilterChain has new/0 instead (no per-sketch options to configure).

serialize(sketch)

@callback serialize(sketch()) :: binary()

Serializes a sketch to its canonical binary representation.

size_bytes(sketch)

@callback size_bytes(sketch()) :: non_neg_integer()

Returns the size, in bytes, of the sketch's serialized state.

update(sketch, item)

(optional)
@callback update(sketch(), item :: term()) :: sketch()

Updates a sketch with a single item, returning the updated sketch.

Optional: not implemented by ExDataSketch.XorFilter (immutable once built).

update_many(sketch, t)

(optional)
@callback update_many(sketch(), Enumerable.t()) :: sketch()

Updates a sketch with every item in an enumerable, returning the updated sketch.

Optional: not implemented by ExDataSketch.XorFilter (immutable once built).

Functions

implemented?(module)

@spec implemented?(module()) :: boolean()

Returns true if module implements every non-optional callback of this behaviour (serialize/1, deserialize/1, size_bytes/1, capabilities/0).

This checks function export, not the @behaviour declaration itself, so it also recognizes modules that satisfy the contract without formally declaring @behaviour ExDataSketch.Sketch.

Examples

iex> ExDataSketch.Sketch.implemented?(ExDataSketch.Bloom)
true

iex> ExDataSketch.Sketch.implemented?(Enum)
false

iex> ExDataSketch.Sketch.implemented?(ExDataSketch.HLL)
true