Buffer partition.
The implementation is based on OpenTelemetry Batch Processor. The use case is very similar. The "OTel batch processor" buffers spans (large/massive amounts of them) and then exports them to an external source after some time (via OTLP). It is designed and implemented for efficiently handling large workloads. The partitioned buffer takes inspiration from the "OTel batch processor" to leverage all these properties.
Double-Buffering Processing Cycle
Phase 1 — Buffering Phase 2 — Processing
=================== ====================
Client writes Timer fires :processing
| |
v v
+-----------+ Allocate a fresh ETS table
| Table A |<-- write and point Tidefall.Metadata
| (current) | at it (writes flow there now)
+-----------+ |
v
+-----------+
| Table B |<-- write (new current)
+-----------+
| Table A |-- give_away --> Task
+-----------+ |
v
:ets.select (batches)
|
processor(batch)
|
batch mode:
:ets.delete(Table A)
:table mode:
processor owns Table A
|
:processing_completedETS access
This module is ETS-agnostic by design. It owns the two backing
tables (allocation, swap, give-away on processing, teardown on
terminate) but delegates the data-shape concerns — record
layout, processor batch shape, conditional-update logic — to the
buffer implementation modules (Tidefall.Queue, Tidefall.HashMap,
…) through the Tidefall.Buffer behaviour callbacks
ets_table_opts/0 and ets_match_spec/0.
Impl modules read the active write table by calling
current_table/1 and then issue their own :ets.* operations.
Summary
Functions
Returns the partition's buffer size.
Returns a specification to start this module under a supervisor.
Returns the partition's current write table.
Starts a buffer partition.
Updates the options for the partition.
Functions
@spec buffer_size(atom()) :: non_neg_integer()
Returns the partition's buffer size.
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec current_table(partition :: atom()) :: :ets.table()
Returns the partition's current write table.
This is the only sanctioned access to the active table.
Buffer implementations call this and then issue their own
:ets.* operations against the returned table reference.
Do not cache
The returned reference is single-use. The partition swaps it on every processing tick; holding it across calls (e.g. caching in a struct, sharing across processes, or pipelining many ETS calls after a stale lookup) is a correctness bug. Resolve it fresh inside each operation.
@spec start_link(keyword()) :: GenServer.on_start()
Starts a buffer partition.
@spec update_options( GenServer.server(), keyword() ) :: :ok
Updates the options for the partition.