RaftEx.LogSegmentWriter (raft_ex v0.1.0)

Background process that manages immutable segment files for long-term log storage.

Segments are immutable files containing a range of log entries. Once a segment is sealed, it cannot be modified. This provides efficient sequential reads and enables compaction of old entries.

Segment File Format

Each segment file contains:

Header: magic bytes, version, first_index, last_index, entry count
Entries: serialized log entries with checksums
Footer: checksum of the entire file

Lifecycle

Active Segment: Receives new entries from WAL
Sealed Segment: Closed when max size/entries reached
Compacted: Merged with other segments during cleanup

Summary

Types

segment()

segment_id()

segment_range()

t()

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

delete_up_to(pid, up_to_index)

Deletes segments up to the given index (used after snapshot).

index_range(pid)

Returns the first and last index across all segments.

overview(pid)

Returns overview of segment state.

read(pid, from_index, to_index)

Reads entries from a segment by index range.

read_entry(pid, index)

Reads a single entry by index.

recover(map)

Recovers segments from disk.

seal_active_segment(pid)

Seals the current active segment and starts a new one.

start_link(config)

Starts the segment writer process.

truncate_from(pid, from_index)

Truncates segments from the given index onwards.

write(pid, entries)

Writes entries to the current active segment or creates a new one.

Types

segment()

@type segment() :: %{
  id: segment_id(),
  path: String.t(),
  range: segment_range(),
  entry_count: non_neg_integer(),
  size_bytes: non_neg_integer(),
  sealed: boolean(),
  fd: reference() | nil
}

segment_id()

@type segment_id() :: non_neg_integer()

segment_range()

@type segment_range() ::
  {first_index :: non_neg_integer(), last_index :: non_neg_integer()}

t()

@type t() :: %RaftEx.LogSegmentWriter{
  active_segment: segment() | nil,
  config: map(),
  data_dir: String.t(),
  max_entries: non_neg_integer(),
  max_size_bytes: non_neg_integer(),
  segments: [segment()]
}

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

delete_up_to(pid, up_to_index)

@spec delete_up_to(pid(), non_neg_integer()) :: :ok | {:error, term()}

Deletes segments up to the given index (used after snapshot).

index_range(pid)

@spec index_range(pid()) :: {non_neg_integer(), non_neg_integer()}

Returns the first and last index across all segments.

overview(pid)

@spec overview(pid()) :: map()

Returns overview of segment state.

read(pid, from_index, to_index)

@spec read(pid(), non_neg_integer(), non_neg_integer()) ::
  {:ok, [RaftEx.Types.log_entry()]} | {:error, term()}

Reads entries from a segment by index range.

read_entry(pid, index)

@spec read_entry(pid(), non_neg_integer()) ::
  {:ok, RaftEx.Types.log_entry()} | {:error, :not_found}

Reads a single entry by index.

recover(map)

@spec recover(map()) :: {:ok, [segment()]} | {:error, term()}

Recovers segments from disk.

seal_active_segment(pid)

@spec seal_active_segment(pid()) :: :ok | {:error, term()}

Seals the current active segment and starts a new one.

start_link(config)

@spec start_link(map()) :: GenServer.on_start()

Starts the segment writer process.

truncate_from(pid, from_index)

@spec truncate_from(pid(), non_neg_integer()) :: :ok | {:error, term()}

Truncates segments from the given index onwards.

write(pid, entries)

@spec write(pid(), [RaftEx.Types.log_entry()]) :: :ok | {:error, term()}

Writes entries to the current active segment or creates a new one.