# Fief v0.1.0 - Table of Contents

> CP key ownership for Elixir clusters: at most one node owns a given key, in every failure mode.

## Pages

- Overview
  - [Overview](overview.md)
  - [Guarantees](guarantees.md)

- Concepts
  - [Ownership](ownership.md)
  - [Leases](leases.md)
  - [Authority](authority.md)
  - [Instances](instances.md)

- Using Fief.Key
  - [Getting started](getting-started.md)
  - [The key lifecycle](key-lifecycle.md)
  - [Fencing modes](fencing-modes.md)
  - [Delivery and errors](delivery-and-errors.md)
  - [Configuration](configuration.md)

- Using Fief.Cache
  - [Using Fief.Cache](cache.md)

- Protocols
  - [Join and leave](join-leave.md)
  - [The transfer protocol](transfer.md)
  - [Failure and recovery](failure.md)
  - [Netsplit matrix](netsplit-matrix.md)

- Operations
  - [Postgres Adapter](postgres.md)
  - [Tuning](tuning.md)
  - [Runbook](runbook.md)

- Extending Fief
  - [Authority Adapters](authority-adapters.md)
  - [Vnode Implementations](vnode-impls.md)
  - [Migration Embedding](migration-embedding.md)

- Internals
  - [Verification](verification.md)
  - [Architecture](architecture.md)

## Modules

- [Fief.Kernel](Fief.Kernel.md): The instance kernel: the `rest_for_one` supervisor over an instance's live
machinery (implementation.md §2). Started as the first child of the thin
outer `Fief.Supervisor`, it holds every process that serves the instance;
the outer wrapper adds only the shutdown-leave sentinel beside it
(`Fief.Node.ShutdownDrain`, `docs/leave-on-shutdown.md` §2.6).
- [Fief.Node.ShutdownDrain](Fief.Node.ShutdownDrain.md): The shutdown-leave sentinel (`docs/leave-on-shutdown.md` §3.2): the last
child of the outer `Fief.Supervisor`, whose only job is to run a graceful
`Fief.Node.leave/1` and wait for the drain to finish while the kernel — the
`Fief.Node`, the planner, and `Fief.Vnode.Manager` — is still alive behind
it.

- Core API
  - [Fief](Fief.md): Consistent vnode ownership for Elixir clusters.
  - [Fief.Cache](Fief.Cache.md): The first-class trivial-cache surface (implementation.md §6.1) — a
partitioned, single-owner cache over the generic `Fief.Vnode`
behaviour and `Fief.Router`. The runtime half is `Fief.Cache.VnodeImpl`;
this module is caller-side: it hashes `key` to a vnode
(`Fief.Hasher`, shared with `Fief.Key`) and calls straight through
`Fief.Router.call/4`.
  - [Fief.Hasher](Fief.Hasher.md): The single pluggable seam for the key-addressed surfaces — `Fief.Key` and
`Fief.Cache` — both of which do caller-side key→vnode hashing over the
generic `Fief.Vnode`/`Fief.Router` layer and must compute the identical
mapping to agree with each other and across nodes. A hasher answers one
question: how does a key term map to an integer used for vnode assignment?
Living here (rather than under `Fief.Key`) keeps `Fief.Cache` from depending
on the key layer for something that is really neutral substrate.
  - [Fief.Hasher.Default](Fief.Hasher.Default.md): The default `Fief.Hasher`: `:erlang.phash2(key, 4_294_967_296)` over
**binaries only**. A non-binary key raises `ArgumentError` (prefixed with the
calling surface's name by `Fief.Hasher.vnode!/4`), telling you to either
encode your keys to binaries yourself or configure a custom `:hasher` module —
the same forced-explicit-canonicalization the old `key_encoder` contract
provided.
  - [Fief.Key](Fief.Key.md): The sanctioned per-key-process layer (design §5.5, §8; implementation.md
§7): hashing, the wrapper message format, the user-facing key behaviour, and
the public key-addressed API. Built purely on the public `Fief.Vnode`
behaviour and `Fief.Router` — the runtime half is `Fief.Key.VnodeImpl`, the
per-key processes are `Fief.Key.Server`, and the ⊨-rule state machines it
embeds are `Fief.Transfer` (the key-generic transfer substrate).

- Authority contracts
  - [Fief.Leadership](Fief.Leadership.md): Leader election contract (design §5.1, implementation.md §3). An adapter is
started per instance and campaigns on behalf of its node; it notifies the
configured pid with
  - [Fief.Leadership.Store](Fief.Leadership.Store.md): The storage primitives a StateStore-colocated leadership seat exposes
(design §5.1's default Leadership implementation — one row, one statement).
`Fief.Leadership.FromStore` is the notify-loop adapter campaigning through
these; any store module implementing them (Fief.Authority.Local today, the
Postgres adapter at M7 — its `fief_leader` UPDATE is exactly `try_lead/3`)
plugs in unchanged. This keeps the adapter generic without coupling it to
`Fief.Authority.Local`: the adapter depends on this two-callback behaviour,
never on a concrete store.
  - [Fief.Presence](Fief.Presence.md): Live-node hints (design §5.1, implementation.md §3). Never load-bearing:
presence accelerates reactions (cache refresh, planner wake-ups) but no
safety property may depend on it. Push messages to subscribers
  - [Fief.StateStore](Fief.StateStore.md): The arbiter contract (design §5.1, implementation.md §3): leases, membership,
the epoch-fenced vnode table, and fencing-token validation.

- Adapters
  - [Fief.Authority.Local](Fief.Authority.Local.md): Single-process, in-memory implementation of the three Authority contracts
(implementation.md §3.2). It exists for three reasons: executable reference
semantics for the contracts, unit tests without Postgres, and single-node
development. **Not for production multi-node use** — it is itself a SPOF
with none of Postgres's durability.
  - [Fief.Authority.Postgres](Fief.Authority.Postgres.md): The Postgres Authority adapter (implementation.md §3.1): `Fief.StateStore`
plus the `Fief.Leadership.Store` primitives, on five namespace-keyed tables
in the user's database (shipped as `Fief.Authority.Postgres.Migrations`).
  - [Fief.Authority.Postgres.Migrations](Fief.Authority.Postgres.Migrations.md): Table definitions for `Fief.Authority.Postgres` (implementation.md §3.1):
five tables, every one keyed by namespace — the instance name, stringified.
One set of migrations covers all instances sharing the database; adding an
instance is rows, never DDL.
  - [Fief.Leadership.FromStore](Fief.Leadership.FromStore.md): The default `Fief.Leadership` adapter (implementation.md §3, design §6.5):
a notify loop campaigning through the `Fief.Leadership.Store` storage
primitives of the instance's StateStore — `try_lead/3` (monotonic term
issuance, TTL-based like leases) and `leader/1`. It is generic over any
store module implementing that two-callback behaviour; today that is
`Fief.Authority.Local`, at M7 the Postgres adapter's `fief_leader` row.

- Vnode layer (experimental)
  - [Fief.Cache.VnodeImpl](Fief.Cache.VnodeImpl.md): The trivial reference `Fief.Vnode` implementation (implementation.md §6.1):
a cache-shaped vnode backed by one ETS table per vnode. It doubles as the
behaviour's documentation example — and it is genuinely trivial
  - [Fief.Vnode](Fief.Vnode.md): The vnode implementation contract (design §5.5, implementation.md §6.1):
key-free and migration-free — ownership and session lifecycle, nothing else.

- Transfer engine
  - [Fief.Transfer](Fief.Transfer.md): The lazy-pull key-migration protocol of design.md §6.3 as two pure state
machines — `Fief.Transfer.Donor` and `Fief.Transfer.Recipient` —
the one place the ⊨ rules are implemented (implementation.md §1 Decision,
§7). Every rule here has a TLC counterexample behind it
(`specs/FiefTransfer.tla`); a third-party vnode impl that wants incremental
migration embeds these machines rather than re-deriving them.
  - [Fief.Transfer.Channel](Fief.Transfer.Channel.md): The blob-transport behaviour (implementation.md §7, Decision): how the
extracted per-key state travels between donor and recipient. v1 ships
in-band (`Fief.Transfer.Channel.InBand`) — the blob rides the `:peer`
payload on dist — behind a hard `max_blob_size`; an out-of-band TCP channel
is the planned fast-follow implementing this same behaviour.
  - [Fief.Transfer.Channel.InBand](Fief.Transfer.Channel.InBand.md): v1 blob transport: the blob rides the `:peer` payload on dist, admitted
only under `ctx.max_blob_size` (default 65_536 — implementation.md §9),
measured as `:erlang.external_size/1`, the actual dist-payload proxy. The
cap is what keeps "dist carries control only" true as a Key-layer
discipline (implementation.md §7): per-key granularity already bounds
messages to one key's state; this bounds one key's state.

  - [Fief.Transfer.Donor](Fief.Transfer.Donor.md): The donor half of the lazy-pull migration (design.md §6.3, steps 3/5/7;
`specs/FiefTransfer.tla` actions `ServePull` / `SweepPush` / `RecvAck` and
the unfreeze half of `RefreshView`). Pure: see `Fief.Transfer` for the
purity contract, the effect vocabulary, and the layer split.
  - [Fief.Transfer.Recipient](Fief.Transfer.Recipient.md): The recipient half of the lazy-pull migration (design.md §6.3, steps 3/4/6;
`specs/FiefTransfer.tla` actions `SendPull` / `RecvPullResp` / `RecvPush` /
`Escheat`). Pure: see `Fief.Transfer` for the purity contract, the
effect vocabulary, and the layer split.

- Internal
  - [Fief.Instance](Fief.Instance.md): Name derivation for instance-scoped processes and tables. Every process and
ETS name in an instance's subtree derives from the instance name, so
instances never collide in one BEAM (implementation.md §2). Bounded atom
minting: one fixed set per instance, created at boot.

  - [Fief.Key.Limbo](Fief.Key.Limbo.md): The per-instance limbo supervision for fence-surviving key servers
(design §8, implementation.md §7 fence modes): a `DynamicSupervisor` plus
the `Fief.Key.Limbo.Tracker` that manages it, supervised at the **instance
root** via `c:Fief.Vnode.instance_children/1` (M7 Decision — previously an
unlinked island). The root is `rest_for_one` and the limbo is its first
child, so it survives crash-restarts of every kernel child (a crashed
`Fief.Node` fences the node — exactly when `:continue` survivors must keep
living) and dies only with the instance. The fence kill itself never
reaches here: it tears down agent link trees under `Fief.Vnode.Manager`.
  - [Fief.Key.Limbo.Tracker](Fief.Key.Limbo.Tracker.md): The bookkeeper half of `Fief.Key.Limbo`: owns the entry map (`{key_id,
vnode_id, pid}` per survivor), starts servers under the sibling
`DynamicSupervisor`, and executes `reap/2`. A plain bookkeeper — its fate
is coupled to that `DynamicSupervisor` under `:one_for_all` (see
`Fief.Key.Limbo`), so a crash of either restarts both together and this
process always comes up beside an empty child set. Its state therefore
never needs reconstruction: there is nothing to rebuild.
  - [Fief.Key.Server](Fief.Key.Server.md): The per-key process (implementation.md §7): a GenServer wrapping the user's
`Fief.Key` module. Started by `Fief.Key.VnodeImpl` from the agent —
`restart: :temporary` (a crashed key process is **not** restarted; the next
message cold-starts, or escheats per the migration machine's
authorization) — under either the impl's agent-linked `DynamicSupervisor`
(`on_fence: :terminate`) or the instance's `Fief.Key.Limbo`
(`:continue`/`:callback` — outside the agent's link tree, so the fence kill
does not reach them; design §8).
  - [Fief.Key.VnodeImpl](Fief.Key.VnodeImpl.md): The sanctioned `Fief.Vnode` implementation (implementation.md §7): a plain
key→pid map with monitors in the impl state (Decision: no `Registry` —
every lookup already happens in the agent, so concurrent-read machinery
buys nothing), key servers under a linked `DynamicSupervisor` started from
`init/2` (`on_fence: :terminate`) or the instance's `Fief.Key.Limbo`
(weaker modes — see `Fief.Key`), and the M5 migration machines embedded
per their moduledoc obligations. Built purely on the public behaviour —
the proof the contract is sufficient.
  - [Fief.Node](Fief.Node.md): The per-node agent (design §5.2, implementation.md §4): a `:gen_statem`
running join/leave, the lease-renewal loop, self-fencing, and the
routing-table cache writer.
  - [Fief.Node.ConfigMismatchError](Fief.Node.ConfigMismatchError.md): Raised at join phase one (design §6.1) when this node's immutable config
disagrees with the namespace's authoritative config. Fatal before any shared
state is touched: the instance refuses to start.

  - [Fief.Node.JoinError](Fief.Node.JoinError.md): Raised when join phase one cannot complete (register/lease failure).
  - [Fief.Planner](Fief.Planner.md): The leader-only fenced rebalance loop (design §5.3, §6.1–6.4;
implementation.md §2). One instance of this process exists per namespace at
a time — on whichever node holds leadership — started and stopped by
`Fief.Planner.Elector` on `:elected`/`:deposed`. Every write is
`cas_assign`/`cas_settle` carrying the planner's term, so a zombie planner
(deposed but not yet notified) is fenced by the StateStore's term ratchet;
duplicate planners can thrash a plan but never corrupt state.
  - [Fief.Planner.Elector](Fief.Planner.Elector.md): Two duties, one always-on process per instance
  - [Fief.Planner.Supervisor](Fief.Planner.Supervisor.md): The planner-hosting subtree (implementation.md §2), one per instance,
sitting between `Fief.Node` and `Fief.Vnode.Manager` under the instance's
`rest_for_one` root. It hosts the `Fief.Leadership` adapter *permanently*
and `Fief.Planner` only while this node holds leadership
  - [Fief.Router](Fief.Router.md): Vnode-level send/`:moved`-retry (design §5.4, implementation.md §5). Pure
module — **no process**: `call/4` and `cast/3` run in the caller. Key
hashing does not exist here; a vnode-level user routes by vnode id directly,
and `Fief.Key` (M6) hashes above this.
  - [Fief.Router.Cache](Fief.Router.Cache.md): The boring ETS owner (implementation.md §2, §5): holds the instance's
routing-cache table. It sits *before* `Fief.Node` under the instance's
`rest_for_one` supervisor, so a `Fief.Node` crash/restart never drops the
cache mid-lookup — the restarted node overwrites rows as it re-joins, and
readers fall back to the Authority on any miss.
  - [Fief.Supervisor](Fief.Supervisor.md): Root supervisor of one Fief instance — a thin **outer** supervisor over two
children (`docs/leave-on-shutdown.md` §2.6)
  - [Fief.Vnode.Agent](Fief.Vnode.Agent.md): One `gen_statem` per owned vnode (implementation.md §6.3): registered under
the bounded-atom name minted by `Fief.Seam.vnode_name/2` (the manager
mints it in its own — sim-bound — process and passes it in), holding
`{impl_module, impl_state}` and running impl callbacks in-process. A vnode
serializes at its agent; impl exceptions crash the agent and the manager's
restart cold-starts it (implementation.md §2).
  - [Fief.Vnode.Manager](Fief.Vnode.Manager.md): The agent lifecycle layer (implementation.md §5.5, §6.3): a small
`one_for_all` supervisor holding the per-instance agent `DynamicSupervisor`
and the reconciliation tracker. Sits AFTER `Fief.Node` under the instance's
`rest_for_one` root, so a `Fief.Node` crash tears the whole vnode layer
down and rebuilds it through the join path.
  - [Fief.Wire](Fief.Wire.md): Framework envelope shapes (implementation.md §8). Three shapes, built and
parsed only here, versioned by a protocol integer

- Seams
  - [Fief.Seam](Fief.Seam.md): The determinism seam (implementation.md §10). Everything nondeterministic —
time, randomness, cross-node communication, and sim-visible progress marks —
goes through this one module, so a test harness can take it over. Together the
seams support a protocol-level deterministic simulation harness
(FoundationDB/TigerBeetle-style): one scheduler owning one event queue.
  - [Fief.Seam.Real](Fief.Seam.Real.md): Production `Fief.Seam`: the real implementations of the function seams.
  - [Fief.Seam.Sim](Fief.Seam.Sim.md): Test-build `Fief.Seam`: real unless hooked (implementation.md §5, §10).
  - [Fief.Sim](Fief.Sim.md): Per-process hook registry for the determinism seams (implementation.md §10).
  - [Fief.Sim.ManualClock](Fief.Sim.ManualClock.md): A steppable virtual clock for tests: time advances only via `advance/2`.
  - [Fief.Sim.Scheduler](Fief.Sim.Scheduler.md): The tier-2 scripted simulation scheduler (implementation.md §10,
build-order M2). Test machinery only: it lives under `lib/fief/sim/` so
test builds of downstream layers can resolve it, but nothing outside
sim/test code may reference it.

