Fief.Vnode behaviour (Fief v0.1.0)

Copy Markdown View Source

The vnode implementation contract (design §5.5, implementation.md §6.1): key-free and migration-free — ownership and session lifecycle, nothing else.

Experimental

v1: experimental. Fief.Key is the stable surface; this contract stabilizes once Fief.Key has been built purely on top of it for a release.

Callbacks run in the Fief.Vnode.Agent process and must return promptly — slow work belongs in linked processes the impl starts from init/2. Everything the impl owns (processes, ETS tables) must live under the agent's link tree, so the fencing kill is complete (implementation.md §2). Impl exceptions crash the agent; the manager restarts it fresh and local state cold-starts — the same recovery path as any process crash.

What the framework enforces around these callbacks (implementation.md §6.2): ownership validation before every handle_message/3, the session envelope before every handle_peer/3, settle only via report_drained/1, and fencing as supervision kill. What it deliberately cannot enforce: the truth of "I hold nothing", escheat gating, taint — those are the impl's obligations (Fief.Transfer for the sanctioned layer).

The trivial reference implementation is Fief.Cache.VnodeImpl: handoff_out → report_drained immediately; handoff_in → start empty. Incremental migration is opt-in via Fief.Transfer (M5/M6), not contract surface.

Agent-provided functions

send_peer/2, report_drained/1 must be called from within an agent callback (they read the agent's bound context); reply/2 may be called from any process — from is self-contained.

Summary

Types

A transfer session: {vnode_id, donor, recipient, epoch} — the epoch is the one produced by the cas_assign that opened the transfer (the row epoch, Fief.StateStore). Sessions make every peer message self-identifying; stale-session messages are dropped by the framework before the impl sees them (⊨ rule: stale inject dropped without ack, design §6.3).

Callbacks

Optional. Joiner-compatibility fingerprint, compared against fief_meta at join phase one; mismatch is fatal before shared state is touched. ctx carries :instance, :partitions, and :opts.

Advisory notification with a deadline; the framework kills the agent (and the linked impl subtree) unconditionally when the deadline passes. Never load-bearing for guarantee 1.

Optional (M6 contract change). Non-framework messages arriving in the agent process: because impl callbacks run in the agent, everything an impl-owned process sends back (extraction completions at freeze), every :DOWN for a monitor the impl holds, and every seam timer the impl armed (Fief.Seam.send_after/2 delivers {:timeout, ref, msg} to the arming process — the agent) lands in the agent's mailbox. This callback is that channel.

Delivered only while this node owns the vnode under a live lease (framework-validated). msg is opaque — a multi-key impl defines a wrapper format (implementation.md §7). from is opaque; reply inline or pass it to an owned process which replies directly via Fief.Vnode.reply/2.

Impl↔impl traffic between successive owners of this vnode. The framework delivers this ONLY when the session matches its authoritative view of the vnode row — stale-session messages are dropped before the impl sees them.

Ownership arrived. session = open transfer with a live donor (pull from it); nil = no donor (fresh assignment, donor dead, or aborted transfer — rebuild from durable truth). The impl keeps its state across handoff transitions, so a donor regaining ownership after recipient death (handoff_out → handoff_in nil) can apply the taint rule from its own memory (implementation.md §7).

Ownership left this node with an open transfer; drain to the recipient via send_peer/2, then report_drained/1. New handle_message deliveries have already stopped (framework validation) before this is called.

Initialize the impl for one vnode. ctx carries :instance, :node_id, :vnode_id, and :opts (the opts half of the vnode_impl: {module, opts} instance opt).

Optional. Child specs the impl needs at the instance root — supervised infrastructure that must outlive fence kills and kernel restarts (the root is rest_for_one; these children are placed first, so they survive a crash of any kernel child and die only with the instance). Fief.Key.VnodeImpl uses this for Fief.Key.Limbo. ctx carries :instance, :partitions, :opts, and :sim.

Functions

Reply to a handle_message/3 from — inline from the agent callback, or later from any impl-owned process. A nil or unrecognized from is a no-op (casts carry no reply path).

The donor's "I hold nothing, for this session" report — the only path to settle (⊨ rule 7's enforcement point, design §6.3). The agent relays it as {:fief_settle_report, session, reporting_node} to the process registered under Fief.Instance.settle_relay_name/1 (in M4 phase A the tests playing planner; in phase B the planner path). Only the session's donor may report; a stale session or a non-donor caller is refused. Call from within an agent callback only.

Send an impl↔impl message to the session's counterpart agent (donor → recipient or recipient → donor), as a Fief.Wire.peer/2 envelope through Fief.Seam. Validated against the agent's current session — the authoritative view recorded at the last handoff observation; a mismatch is {:error, :stale_session} and nothing is sent. Call from within an agent callback only.

Types

impl_state()

@type impl_state() :: term()

session()

@type session() ::
  {vnode_id(), donor :: node(), recipient :: node(), epoch :: pos_integer()}

A transfer session: {vnode_id, donor, recipient, epoch} — the epoch is the one produced by the cas_assign that opened the transfer (the row epoch, Fief.StateStore). Sessions make every peer message self-identifying; stale-session messages are dropped by the framework before the impl sees them (⊨ rule: stale inject dropped without ack, design §6.3).

vnode_id()

@type vnode_id() :: non_neg_integer()

Callbacks

config_fingerprint(ctx)

(optional)
@callback config_fingerprint(ctx :: map()) :: term()

Optional. Joiner-compatibility fingerprint, compared against fief_meta at join phase one; mismatch is fatal before shared state is touched. ctx carries :instance, :partitions, and :opts.

handle_fence(deadline_ms, impl_state)

@callback handle_fence(deadline_ms :: non_neg_integer(), impl_state()) ::
  {:ok, impl_state()}

Advisory notification with a deadline; the framework kills the agent (and the linked impl subtree) unconditionally when the deadline passes. Never load-bearing for guarantee 1.

handle_info(msg, impl_state)

(optional)
@callback handle_info(msg :: term(), impl_state()) :: {:noreply, impl_state()}

Optional (M6 contract change). Non-framework messages arriving in the agent process: because impl callbacks run in the agent, everything an impl-owned process sends back (extraction completions at freeze), every :DOWN for a monitor the impl holds, and every seam timer the impl armed (Fief.Seam.send_after/2 delivers {:timeout, ref, msg} to the arming process — the agent) lands in the agent's mailbox. This callback is that channel.

Routing rule — the framework claims, and the impl never sees:

  • {:fief_fence, deadline_ms} from the manager (and any future framework-internal message, which will use a fief_agent_-prefixed tag — that tag prefix is reserved);
  • {:timeout, ref, msg} for timers the agent itself armed — claimed by ref, never by shape. The agent arms no timers today, so in practice every {:timeout, ref, msg} is delivered here whole (match your own refs; ignore the rest — the stale-timer catch-all is yours now);
  • anything that decodes as a Fief.Wire envelope (:msg/:peer take the validated paths above; :moved and unsupported protocol versions are dropped);
  • :gen_statem calls (:reconcile and friends).

Everything else is delivered here whenever the impl exists (always, after init/2 — both :serving and :donor postures), under the same checkpoint discipline as every other delivery. Not exported → such messages are dropped silently (the pre-M6 behavior).

handle_message(msg, from, impl_state)

@callback handle_message(msg :: term(), from :: term() | nil, impl_state()) ::
  {:noreply, impl_state()} | {:reply, term(), impl_state()}

Delivered only while this node owns the vnode under a live lease (framework-validated). msg is opaque — a multi-key impl defines a wrapper format (implementation.md §7). from is opaque; reply inline or pass it to an owned process which replies directly via Fief.Vnode.reply/2.

handle_peer(session, msg, impl_state)

@callback handle_peer(session(), msg :: term(), impl_state()) :: {:noreply, impl_state()}

Impl↔impl traffic between successive owners of this vnode. The framework delivers this ONLY when the session matches its authoritative view of the vnode row — stale-session messages are dropped before the impl sees them.

handoff_in(arg1, impl_state)

@callback handoff_in(session() | nil, impl_state()) :: {:ok, impl_state()}

Ownership arrived. session = open transfer with a live donor (pull from it); nil = no donor (fresh assignment, donor dead, or aborted transfer — rebuild from durable truth). The impl keeps its state across handoff transitions, so a donor regaining ownership after recipient death (handoff_out → handoff_in nil) can apply the taint rule from its own memory (implementation.md §7).

handoff_out(session, impl_state)

@callback handoff_out(session(), impl_state()) :: {:ok, impl_state()}

Ownership left this node with an open transfer; drain to the recipient via send_peer/2, then report_drained/1. New handle_message deliveries have already stopped (framework validation) before this is called.

init(vnode_id, ctx)

@callback init(vnode_id(), ctx :: map()) :: {:ok, impl_state()}

Initialize the impl for one vnode. ctx carries :instance, :node_id, :vnode_id, and :opts (the opts half of the vnode_impl: {module, opts} instance opt).

instance_children(ctx)

(optional)
@callback instance_children(ctx :: map()) :: [
  Supervisor.child_spec() | {module(), term()}
]

Optional. Child specs the impl needs at the instance root — supervised infrastructure that must outlive fence kills and kernel restarts (the root is rest_for_one; these children are placed first, so they survive a crash of any kernel child and die only with the instance). Fief.Key.VnodeImpl uses this for Fief.Key.Limbo. ctx carries :instance, :partitions, :opts, and :sim.

Functions

reply(arg1, reply)

@spec reply(term(), term()) :: :ok

Reply to a handle_message/3 from — inline from the agent callback, or later from any impl-owned process. A nil or unrecognized from is a no-op (casts carry no reply path).

report_drained(session)

@spec report_drained(session()) :: :ok | {:error, :stale_session | :not_donor}

The donor's "I hold nothing, for this session" report — the only path to settle (⊨ rule 7's enforcement point, design §6.3). The agent relays it as {:fief_settle_report, session, reporting_node} to the process registered under Fief.Instance.settle_relay_name/1 (in M4 phase A the tests playing planner; in phase B the planner path). Only the session's donor may report; a stale session or a non-donor caller is refused. Call from within an agent callback only.

send_peer(session, msg)

@spec send_peer(session(), term()) ::
  :ok | {:error, :stale_session | :noconnect | :nosuspend}

Send an impl↔impl message to the session's counterpart agent (donor → recipient or recipient → donor), as a Fief.Wire.peer/2 envelope through Fief.Seam. Validated against the agent's current session — the authoritative view recorded at the last handoff observation; a mismatch is {:error, :stale_session} and nothing is sent. Call from within an agent callback only.