Ferricstore.Store.Shard (ferricstore v0.3.1)

Copy Markdown View Source

GenServer managing one Bitcask partition backed by an ETS hot-read cache.

v2 Architecture: Pure Stateless NIFs

All state lives in Elixir (ETS keydir + GenServer state). Rust NIFs are pure stateless functions: v2_append_record, v2_pread_at, v2_fsync, v2_append_batch, v2_append_tombstone, v2_scan_file, hint file I/O. No Rust-side Store resource, HashMap keydir, or Mutex.

Write path: group commit

  1. The key is written to ETS immediately (reads see it at once).
  2. The entry is appended to an in-memory pending list.
  3. A recurring :drain_pending timer fires every @flush_interval_ms and calls NIF.v2_append_batch_nosync/2 with all accumulated entries, then updates ETS entries with their disk locations (file_id, offset, value_size). This step moves bytes from BEAM memory to the kernel page cache — no fsync. Data-file durability is owned by Ferricstore.Store.BitcaskCheckpointer, which runs on its own, longer tick and issues v2_fsync against the same active file.
  4. File rotation occurs when the active file exceeds 256 MB.

Read path: ETS bypass

Router.get/1 and Router.get_meta/1 read ETS directly without going through this GenServer for hot (cached) keys. Cold keys (value=nil in ETS) have their disk location (file_id, offset) stored in the ETS 7-tuple, enabling direct v2_pread_at without scanning.

ETS layout

Each entry is a 7-tuple {key, value, expire_at_ms, lfu_counter, file_id, offset, value_size} where expire_at_ms = 0 means the key never expires. The file_id, offset, and value_size fields enable cold reads without scanning, STRLEN on cold keys, and sendfile zero-copy. Expired entries are lazily evicted on read.

Process registration

Shards register under the name returned by Ferricstore.Store.Router.shard_name/1, e.g. :"Ferricstore.Store.Shard.0".

Summary

Functions

Returns a specification to start this module under a supervisor.

Starts a shard GenServer.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Starts a shard GenServer.

Options

  • :index (required) -- zero-based shard index
  • :data_dir (required) -- base directory for Bitcask data files
  • :flush_interval_ms -- batch-commit interval in ms (default: 1)