Normandy.Agents.Turn.ResumeReaper (normandy v2.2.0)

View Source

Eager-handoff trigger for distributed sessions.

Horde.DynamicSupervisor does NOT redistribute a node's children when that node dies (with members: :auto the dead member is removed from the CRDT, so the reclaim path never fires โ€” see the Phase-7 design ยง7.6). This reaper provides selective eager handoff instead: it monitors cluster nodes and, on :nodedown, restarts the :eager sessions whose server died with the lost node.

A session is reaped when (1) its turn is in flight (SessionStore.list_resumable/1 lists the sessions whose last turn-state save was :eager and unfinished), (2) it is not registered anywhere (SessionRegistry.whereis/2 == :none), and (3) its persisted turn state is still non-terminal (this guards against an index member a crash left behind). A deleted session (Turn.Session.delete/1) is never listed. It is restarted as a thin spec under the supervisor; the started Turn.Server reconstructs config from the persisted template and Turn.resume/2s the in-flight turn โ€” no inbound request.

The restarted server restores the plain-data options saved with the template (:approval_timeout_ms, :on_approval_timeout, :idle_timeout_ms). Functions and pids can't be saved, so pass :subscriber and :handlers to the reaper; it forwards them to every session it restarts. Without them, a revived session runs the default handlers and has no subscriber to tell about a pending approval.

Run one reaper per node (host supervision tree). Concurrent reapers on multiple survivors are safe: the registry's atomic :via registration means only one start wins; a loser's {:already_started, _} is treated as a successful start (the shared Turn.Start.start_server/6 maps it to {:ok, pid}), so nothing is logged or retried.

Summary

Functions

Returns a specification to start this module under a supervisor.

Restart every eager, unregistered, non-terminal session under the supervisor. Exposed (not private) so it can be unit-tested without orchestrating real nodes.

Starts the reaper.

Types

opts()

@type opts() :: [
  name: GenServer.name(),
  store: {module(), term()},
  registry: {module(), term()},
  supervisor: term(),
  supervisor_mod: module(),
  template_provider: {module(), term()},
  subscriber: (atom(), map() -> any()) | nil,
  handlers: Normandy.Agents.Turn.Driver.Handlers.t() | nil
]

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

reap(state)

@spec reap(map()) :: :ok

Restart every eager, unregistered, non-terminal session under the supervisor. Exposed (not private) so it can be unit-tested without orchestrating real nodes.

Sessions are checked and started concurrently, up to 16 at a time; reap/1 returns once every one of them is done. A session whose check or start raises, exits or throws is logged by the failure's type and skipped; the others are still reaped.

Emits one [:normandy, :resume_reaper, :sweep] event per call (see Normandy.Telemetry).

start_link(opts)

@spec start_link(opts()) :: GenServer.on_start()

Starts the reaper.

Options

  • :name (term/0) - A GenServer.name/0 for the reaper.

  • :store - Required. The SessionStore as {module, handle}; its list_resumable/1 names the sessions with an eager turn in flight.

  • :registry - Required. The SessionRegistry as {module, handle}.

  • :supervisor (term/0) - Required. The supervisor sessions restart under (its name, for Horde).

  • :supervisor_mod (module/0) - A Normandy.Agents.Turn.ServerSupervisor whose start_server/2 restarts a session; checked as Turn.Session.new/1 checks its :supervisor_mod. The default value is Normandy.Agents.Turn.Supervisor.Horde.

  • :template_provider ({module, handle}) - Required. The AgentTemplate as {module, handle} (thin starts).

  • :subscriber - Passed to every session the reaper restarts.

  • :handlers (a Normandy.Agents.Turn.Driver.Handlers or nil) - Passed to every session the reaper restarts.