Authority

Copy Markdown View Source

For users and operators who want the mental model of how Fief decides who owns what. Assumes the guarantees; no Postgres or protocol detail needed.

Every safety property in Fief traces back to one place: a small, linearizable store that Fief calls the arbiter. Nodes do not vote, gossip, or negotiate ownership among themselves. Each node derives its behavior from the arbiter's answers and from its own authoritative knowledge of itself — its live lease and the vnodes it owns. When two nodes disagree, they do not reconcile; they both re-read the arbiter, and the arbiter is right.

The arbiter is not one monolithic dependency. It is three contracts, each trusted for a different thing, each consulted at a different frequency, and each independently pluggable. The v1 Postgres adapter satisfies the two that carry a safety obligation — StateStore and Leadership — from one database, and nothing above those two contracts can tell they share a table, which is the test that the split is drawn correctly. Presence has no shipped adapter yet (below) — a gap the design tolerates precisely because Presence is trusted for nothing.

The three contracts

StateStore — the safety core

Fief.StateStore is the only contract any safety property depends on. It is a linearizable store for three resource families:

  • Leases — acquire/renew/release, plus read_leases (the planner's authoritative liveness read). A lease is a TTL grant judged on the arbiter's clock. This is the record fencing is defined against.
  • Membership — register/set_status/read_members, the joining → active → leaving lifecycle, plus ensure_config for the first-writer-wins immutable namespace config checked at join.
  • The vnode table — read_table/cas_assign/cas_settle. Ownership is derived from this table (owner(key) = table[hash(key)]); see ownership. Every write is a compare-and-swap guarded by both an epoch and a leader term, and every successful write bumps the epoch — there are no non-bumping writes.

What StateStore is trusted for: everything. At most one live holder per lease; epoch and term monotonic; no write with a stale fencing token ever succeeds. These are the invariants the guarantees rest on, so this is the contract whose semantics are pinned by an executable spec rather than by prose. The spec is Fief.StateStoreContract (test/support/state_store_contract.ex) — an adapter is only a StateStore if it passes it verbatim, and the Postgres adapter does. Rather than restate its assertions, this page names the blocks that carry each claim:

  • Lease liveness and the arbiter-clock discipline — describe "StateStore contract: leases".
  • Epoch monotonicity across membership and table writes — describe "StateStore contract: membership and epoch".
  • CAS fencing on epoch and the term ratchet against zombie planners — describe "StateStore contract: vnode table CAS".
  • First-writer-wins config — describe "StateStore contract: namespace config".

:unreachable is not :expired. The lease callbacks distinguish "I could not reach the arbiter" ({:error, :unreachable}) from "the arbiter says your lease is dead" ({:error, :expired}), and Fief.Node acts on them differently: unreachable starts the self-fence clock, expired fences now. An unreachable arbiter never expires anything on its own — the lease stays live at the arbiter until its TTL runs out there. This distinction is part of the contract, exercised by the unreachable is distinguishable from expired test inside describe "StateStore contract: leases". It is the seam that lets a node partitioned from the arbiter self-fence before its TTL while peers wait for arbiter-judged expiry — see leases.

Leadership — selects the planner, occasionally

Fief.Leadership picks the single node that runs the rebalancing planner and hands it a monotonic term. It notifies its node with {:fief_leadership, :elected, term} / {:fief_leadership, :deposed} and answers leader/1 on demand.

What Leadership is trusted for: term monotonicity, and nothing else. Leader uniqueness is explicitly not required for safety, because every planner write is fenced by term in the StateStore — a deposed or momentarily-duplicated planner cannot corrupt state; its writes fail at the CAS. Uniqueness is a liveness nicety, not a safety input. Because Leadership gates only fenced writes, it needs none of the lease layer's clock conservatism: no self-fencing, no drift margins, election as eager as the mechanism allows. A leaderless interval costs a pause in rebalancing, never correctness.

The shipped implementation is a StateStore row driven through the Fief.Leadership.Store primitives (try_lead/leader), selected with leadership: :from_authority (the default; false disables campaigning on a node). The contract is drawn so the same seat could be an external term source — a Kafka consumer-group generation, for example — but no alternative ships today; docs/design.md §5.1 records the intended design space.

Grounding: Fief.Leadership and Fief.Leadership.Store define the signatures; there is no separate Leadership contract suite in v1, so term monotonicity is enforced where it is load-bearing — the term ratchet in describe "StateStore contract: vnode table CAS" (test/support/state_store_contract.ex). Design notes: docs/design.md §5.1, §6.5.

Presence — the liveness shadow, hint-grade by construction

Fief.Presence reports the current set of live nodes and optionally pushes {:fief_presence, :up | :down, node_id}. Its whole job is to make other components re-read authoritative state sooner: the planner hears :down and checks that node's lease immediately; the router refreshes its table.

What Presence is trusted for: nothing. This is deliberate and enforced by the shape of the interface — the active set carries no term and no identity, so there is nothing to compare against and nothing to be tempted to act on directly. When presence and leases disagree, the lease wins, and the disagreement means only "go look at the leases sooner." If the presence channel is severed, reactions slow from push tempo to poll tempo and nothing else changes.

v1 ships no Presence adapter for any deployment. Fief.Authority.Local implements the contract for tests and simulation; nothing wires an equivalent for Fief.Authority.Postgres yet, so a production instance today runs permanently at poll tempo rather than push tempo — Fief.Node and the planner both already handle {:fief_presence, ...} hints if one arrives, there is simply nothing sending them outside tests. This is safe by the same argument as a severed channel — Presence is trusted for nothing — but it does mean "poll tempo" is the running default, not a degraded case you might never see.

Grounding: Fief.Presence defines the signature; it authorizes nothing, so there is no safety claim to pin to a contract. Its harmlessness-when-severed is a partition case — see the presence-severed row of the netsplit matrix. Design notes: docs/design.md §5.1, §6.6.

The consultation gradient

The three contracts form a gradient, and the gradient is the point: the availability a layer demands shrinks with how rarely its answer is needed.

  • StateStore — consulted constantly, for correctness. Leases renew on a tight interval; every ownership change is a CAS here. This is the contract that must be real, durable, linearizable storage. In v1, StateStore down freezes the cluster within a lease TTL: unavailable, never inconsistent.
  • Leadership — consulted occasionally, and cheaply fenced. Only when the planner needs to act. Down means no rebalancing until it returns; routing, leases, and self-fencing are unaffected.
  • Presence — ticked constantly, load-bearing never. A heartbeat, not a dependency. Nothing stops serving when it stops.

Nodes do not consult the arbiter on the message hot path at all. Routing a message to a key uses only the node's cached copy of the vnode table; the arbiter is touched for leases, membership, and rebalancing — never per message. That is what lets Fief scale in keys rather than nodes (see the guarantees).

Hints accelerate, never authorize

One asymmetry underlies the whole design, and it is worth stating on its own:

Hints accelerate, never authorize. Receivers hold truth; senders hold hints.

A node's cached vnode table, a nodedown signal, a planner refresh ping, a {:moved, ...} reply — all of these are hints. Every one of them can be stale, lost, or severed, and the system stays correct; it only reacts more slowly. No hint is ever trusted on its own to authorize an action.

The load-bearing half is the sender/receiver split. When a node sends a message for a key, it routes on its own cached table — a hint, possibly stale. The receiving node is the one party that authoritatively knows its own lease state and which vnodes it owns, so the receiver validates every inbound message against its own truth. A message that arrives at a node that no longer owns the vnode is answered {:moved, epoch, delta}, not executed; the sender patches its cache and retries. There is no synchronized table distribution, no global barrier on rebalance, and no way for a stale sender to cause a wrong node to act — because the stale party is never the one who decides.

This is why convergence is a performance property in Fief, not a safety property: every hint channel in the system can be cut and the answers stay correct.

Design notes: docs/design.md §4 ("Hints accelerate, never authorize" and "Receivers hold truth; senders hold hints"), §5.4. The one boundary that holds regardless of deployment: Presence and any external system behind it supply membership hints, never leases — self-fencing requires expiry judged against the clock of the party that grants it, so lease grant and failure detection may never be split across two systems.

Configuring the authority

The authority is chosen per instance with the authority option, and it defaults to {Fief.Authority.Local, []}:

# Production: Postgres plays all three roles from one database.
authority: {Fief.Authority.Postgres, repo: MyApp.Repo}

# Default when unset — single-node development only.
authority: {Fief.Authority.Local, []}

Fief.Authority.Local is a single in-memory process implementing all three contracts. It is the executable reference for the contract semantics and the substrate for tests and single-node iex -S mix development — but it is a single point of failure with none of Postgres's durability, and it carries its own not-for-production banner in its moduledoc. Do not run Local in a multi-node deployment.

Fief.Authority.Postgres is the production adapter: Fief.StateStore plus the Fief.Leadership.Store primitives, over five namespace-keyed tables in your own database (shipped as Fief.Authority.Postgres.Migrations). Every operation is a single statement — PgBouncer transaction mode is the native grain — and every lease and leadership expiry compares against the database's clock, never a node's. It does not implement Fief.Presence — see above: v1 has no shipped Presence adapter for Postgres, so a Postgres instance runs at poll tempo throughout, not only when a hint channel happens to be down. Setup lives in operations/postgres.

The three contracts are drawn independently so that backends can eventually be mixed — a Kafka-driven Leadership over a Postgres StateStore, say — without touching anything above the Authority. Today the choice is the authority adapter itself: leadership accepts only :from_authority (the default) or false, and there is no separate presence or leadership backend option (configuration is the authoritative option list; docs/design.md §5.1 records the design space).

What this means for you

  • There is exactly one source of truth, and it is not your application code. Ownership is derived from the arbiter's vnode table; you never assert ownership, you observe it. A buggy key module — or any vnode implementation — cannot break routing single-ownership, because validation happens on the receiver, outside your code.
  • Only the StateStore has to be highly available for safety. Leadership and Presence can be down, slow, or wrong and the worst you get is delayed rebalancing or slower reactions — never a second owner. Budget your operational attention accordingly.
  • StateStore down means unavailable, not inconsistent. Within a lease TTL, nodes that cannot reach the arbiter stop serving. Plan for the downtime; you never have to plan for split-brain. The netsplit matrix spells out each case.
  • Use Postgres in production; Local is for one node. The default is Local precisely because it is zero-config for development — but shipping it to a cluster gives you a single in-memory SPOF with no durability.
  • Stale is safe. Your nodes route on cached, possibly-stale tables all the time by design. A stale route costs a :moved bounce and a retry, never a wrong execution — so you never need to force cache freshness for correctness.