Capability-aware composition framework for chaining membership filters.
FilterChain composes membership filter structures into ordered query pipelines. It enables lifecycle-tier patterns such as a hot Cuckoo filter (absorbing writes) followed by a cold XorFilter (compacted snapshot).
Chain Roles
Each filter type has valid chain positions:
- Front/Middle/Terminal: Bloom, Cuckoo, Quotient, CQF -- dynamic filters that support member? and put.
- Terminal only: XorFilter -- static, no incremental insert. Must be the last query stage.
- Adjunct only: IBLT -- reconciliation helper, not in the query path.
Query Semantics
member?/2 evaluates query stages in order with short-circuit semantics:
a definite "no" from any stage returns false immediately. Adjuncts are
never queried.
Insert Semantics
put/2 forwards to all query stages that support :put, skipping static
stages (XorFilter). Returns {:ok, chain} or {:error, :full} if a
Cuckoo stage is full.
Delete Semantics
delete/2 checks that ALL query stages support :delete. If any stage
lacks delete support (e.g., Bloom), raises UnsupportedOperationError.
Binary Format (FCN1)
FilterChain serializes each stage independently using its own serialize/1
(always the default :v2 format -- FilterChain does not currently thread
a :format option through to its stages), wrapped in a chain manifest
with magic bytes "FCN1". This is a bespoke container format, not an
ExDataSketch.Codec/EXSK frame -- it has no Codec.sketch_id of its
own. The format: :v1 escape hatch available on every other sketch
family (opt-in legacy EXSK v1 output for v0.7.x readers, see e.g.
ExDataSketch.Bloom.serialize/2) therefore does not apply to
FilterChain.serialize/1 itself.
Summary
Functions
Adds a filter stage to the chain.
Returns the list of adjunct stages.
Returns the set of capabilities supported by FilterChain.
Returns the sum of counts across all query stages.
Deletes an item from all query stages.
Deserializes an FCN1 binary into a FilterChain.
Tests whether an item may be a member by querying all stages in order.
Creates an empty FilterChain.
Inserts an item into all query stages that support :put.
Serializes the FilterChain to the FCN1 binary format.
Callback implementation for ExDataSketch.Sketch.size_bytes/1.
Returns the list of query stages.
Inserts an item into all query stages that support :put, raising if a
stage is full.
Inserts every item in an enumerable via update/2, raising if a stage
fills up partway through.
Types
Functions
Adds a filter stage to the chain.
The stage is automatically classified based on its module type:
- IBLT goes to the adjuncts list (not in query path)
- XorFilter is appended as a terminal query stage
- All other filters are appended as query stages
Raises InvalidChainCompositionError if the composition is invalid
(e.g., adding a query stage after a XorFilter terminal).
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> chain = ExDataSketch.FilterChain.add_stage(chain, ExDataSketch.Bloom.new(capacity: 100))
iex> length(ExDataSketch.FilterChain.stages(chain))
1
Returns the list of adjunct stages.
Examples
iex> ExDataSketch.FilterChain.adjuncts(ExDataSketch.FilterChain.new())
[]
Returns the set of capabilities supported by FilterChain.
@spec count(t()) :: non_neg_integer()
Returns the sum of counts across all query stages.
Examples
iex> ExDataSketch.FilterChain.count(ExDataSketch.FilterChain.new())
0
Deletes an item from all query stages.
Raises UnsupportedOperationError if any query stage does not support
:delete (e.g., Bloom or XorFilter).
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> cuckoo = ExDataSketch.Cuckoo.new()
iex> {:ok, cuckoo} = ExDataSketch.Cuckoo.put(cuckoo, "hello")
iex> chain = ExDataSketch.FilterChain.add_stage(chain, cuckoo)
iex> {:ok, chain} = ExDataSketch.FilterChain.delete(chain, "hello")
iex> ExDataSketch.FilterChain.member?(chain, "hello")
false
@spec deserialize(binary()) :: {:ok, t()} | {:error, Exception.t()}
Deserializes an FCN1 binary into a FilterChain.
Returns {:ok, chain} on success or {:error, reason} on failure.
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> chain = ExDataSketch.FilterChain.add_stage(chain, ExDataSketch.Bloom.new(capacity: 100))
iex> binary = ExDataSketch.FilterChain.serialize(chain)
iex> {:ok, recovered} = ExDataSketch.FilterChain.deserialize(binary)
iex> length(ExDataSketch.FilterChain.stages(recovered))
1
Tests whether an item may be a member by querying all stages in order.
Short-circuits on the first false result. Adjuncts are not queried.
Returns false if the chain has no query stages.
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> bloom = ExDataSketch.Bloom.new(capacity: 100) |> ExDataSketch.Bloom.put("hello")
iex> chain = ExDataSketch.FilterChain.add_stage(chain, bloom)
iex> ExDataSketch.FilterChain.member?(chain, "hello")
true
@spec new() :: t()
Creates an empty FilterChain.
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> ExDataSketch.FilterChain.stages(chain)
[]
Inserts an item into all query stages that support :put.
Skips static stages (XorFilter). Returns {:ok, chain} on success
or {:error, :full} if a Cuckoo stage is full.
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> chain = ExDataSketch.FilterChain.add_stage(chain, ExDataSketch.Bloom.new(capacity: 100))
iex> {:ok, chain} = ExDataSketch.FilterChain.put(chain, "hello")
iex> ExDataSketch.FilterChain.member?(chain, "hello")
true
Serializes the FilterChain to the FCN1 binary format.
Each stage is serialized independently using its own serialize/1,
then wrapped in a chain manifest.
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> chain = ExDataSketch.FilterChain.add_stage(chain, ExDataSketch.Bloom.new(capacity: 100))
iex> binary = ExDataSketch.FilterChain.serialize(chain)
iex> <<"FCN1", _rest::binary>> = binary
iex> byte_size(binary) > 0
true
@spec size_bytes(t()) :: non_neg_integer()
Callback implementation for ExDataSketch.Sketch.size_bytes/1.
Returns the list of query stages.
Examples
iex> ExDataSketch.FilterChain.stages(ExDataSketch.FilterChain.new())
[]
Inserts an item into all query stages that support :put, raising if a
stage is full.
Added so ExDataSketch.FilterChain satisfies the ExDataSketch.Sketch
behaviour's generic update/2 callback, which requires a bare-struct
return. put/2 (returning {:ok, chain} | {:error, :full}) remains the
family-idiomatic name for callers who need to detect a full stage.
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> chain = ExDataSketch.FilterChain.add_stage(chain, ExDataSketch.Bloom.new(capacity: 100))
iex> chain = ExDataSketch.FilterChain.update(chain, "hello")
iex> ExDataSketch.FilterChain.member?(chain, "hello")
true
@spec update_many(t(), Enumerable.t()) :: t()
Inserts every item in an enumerable via update/2, raising if a stage
fills up partway through.
Added so ExDataSketch.FilterChain satisfies the ExDataSketch.Sketch
behaviour's generic update_many/2 callback.
Examples
iex> chain = ExDataSketch.FilterChain.new()
iex> chain = ExDataSketch.FilterChain.add_stage(chain, ExDataSketch.Bloom.new(capacity: 100))
iex> chain = ExDataSketch.FilterChain.update_many(chain, ["a", "b", "c"])
iex> ExDataSketch.FilterChain.member?(chain, "a")
true