Accumulates points per series and writes gorilla-compressed segments to file-based storage (ShardStore).
Each SegmentBuilder owns its own shard directory. Points arrive in batches from the paired Buffer shard. The SegmentBuilder groups them into time-bounded segments (default 4h), compresses with GorillaStream + zstd, and writes to ShardStore (WAL + immutable .seg files).
Tier chunks, watermarks, and raw segments are all in ShardStore. No SQLite is used for shard data.
Summary
Functions
Returns a specification to start this module under a supervisor.
Compact a tier's chunks.dat by rewriting only live entries. Goes through GenServer.
Delete raw segments with end_time before cutoff. Goes through GenServer.
Delete tier chunks where chunk_end < cutoff. Goes through GenServer.
Force flush all open segments to disk.
Ingest a batch of grouped points. Called by Buffer shards after flush.
Synchronous ingest. Used during shutdown to ensure data is received before termination.
Get the count of points held in memory (not yet in a finalized segment).
Get distinct series_ids present in raw storage. Lock-free.
Get aggregate stats for raw segments in this shard. Lock-free.
Read raw segments for ALL series within a time range (used by rollup). Lock-free.
Read the latest raw segment for a series. Lock-free.
Read raw segments for a specific series within a time range. Lock-free.
Read a single tier chunk by exact key (for rollup merge). Lock-free.
Read tier chunks for a specific series within a time range. Lock-free.
Read tier chunks for ALL series in a time range (for tier-to-tier rollup). Lock-free.
Read the latest tier chunk for a series. Lock-free.
Get distinct series_ids from a tier. Lock-free.
Get aggregate stats for a tier. Lock-free.
Read a watermark value for a tier. Lock-free (ETS lookup).
Calculate dead bytes in a tier's chunks.dat. Lock-free.
Write a batch of tier chunks. Goes through GenServer for write serialization.
Write a watermark value for a tier. Goes through GenServer.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.
Compact a tier's chunks.dat by rewriting only live entries. Goes through GenServer.
Options
:threshold- minimum dead/total ratio to trigger (default: 0.3)
Returns {:ok, reclaimed_bytes} or :noop.
Delete raw segments with end_time before cutoff. Goes through GenServer.
Delete tier chunks where chunk_end < cutoff. Goes through GenServer.
Force flush all open segments to disk.
Ingest a batch of grouped points. Called by Buffer shards after flush.
Synchronous ingest. Used during shutdown to ensure data is received before termination.
Get the count of points held in memory (not yet in a finalized segment).
Get distinct series_ids present in raw storage. Lock-free.
Returns {:ok, [[series_id], ...]}.
Get aggregate stats for raw segments in this shard. Lock-free.
Returns %{segment_count, total_points, raw_bytes, oldest_ts, newest_ts}.
Read raw segments for ALL series within a time range (used by rollup). Lock-free.
Returns {:ok, [[series_id, start_time, end_time, data], ...]} sorted by (series_id, start_time).
Read the latest raw segment for a series. Lock-free.
Returns {:ok, [[data]]} or {:ok, []}.
Read raw segments for a specific series within a time range. Lock-free.
Returns {:ok, [[data, start_time, end_time], ...]} sorted by start_time.
Read a single tier chunk by exact key (for rollup merge). Lock-free.
Returns the blob binary, or nil if not found.
Read tier chunks for a specific series within a time range. Lock-free.
Returns {:ok, [[data], ...]} sorted by chunk_start.
Read tier chunks for ALL series in a time range (for tier-to-tier rollup). Lock-free.
Returns {:ok, [[series_id, data], ...]}.
Read the latest tier chunk for a series. Lock-free.
Returns {:ok, [[data]]} or {:ok, []}.
Get distinct series_ids from a tier. Lock-free.
Returns {:ok, [[series_id], ...]}.
Get aggregate stats for a tier. Lock-free.
Returns {chunks, buckets, compressed_bytes}.
Read a watermark value for a tier. Lock-free (ETS lookup).
Returns the integer watermark value, or 0 if not set.
Calculate dead bytes in a tier's chunks.dat. Lock-free.
Returns {dead_bytes, total_file_bytes}.
Write a batch of tier chunks. Goes through GenServer for write serialization.
entries = [{series_id, chunk_start, chunk_end, bucket_count, blob}, ...]
Write a watermark value for a tier. Goes through GenServer.