View Source ProcessHub.Service.Recovery (ProcessHub v0.7.0)
Orphan reconcile: every round converges the cluster toward the hub's declared
list (ProcessHub.Service.DeclaredChildren) — it starts
declared − observed running anywhere through the normal start path with
check_existing: true, and stops running children whose declared entry was
removed. The same difference covers a whole-cluster restart and a single-node
rejoin; stop knowledge is list absence and never expires.
Experimental
The orphan reconcile (the
:auto_recoverylifecycle) is experimental and may change in future releases. Use in production at your own discretion.
The first round runs reconcile_grace_ms after coordinator start, later
rounds follow completed synchronisation rounds, rate-limited to one per
reconcile_interval_ms. This module owns the :auto_recovery config, the
scheduling, and the recovery lifecycle; the round itself lives in
ProcessHub.Service.Recovery.Round and the coordinator stays the GenServer.
See guides/Persistence.md for the model.
Summary
Functions
Blocks until the coordinator reaches :normal or timeout_ms elapses.
Completes the first round: moves the coordinator to :normal, dispatches the
transition hook, and fires the async post_recovery_replay.
Returns the parsed :auto_recovery config for a settings struct, falling back to
the disabled config for any shape the coordinator would reject.
Returns the disabled (default) config.
Parses the :auto_recovery config field into a normalized map.
Deprecated. Armed the next boot for marker-driven replay by deleting the local marker file.
Deprecated. Fanned prepare_recovery/1 out to every hub member.
Returns the coordinator's current :recovery_state.
Returns whether a round triggered by a completed synchronisation round may run.
Schedules the first reconcile round reconcile_grace_ms after coordinator start.
Runs a round in a separate process; replies to the coordinator with
{:reconcile_done, result}. The reply is what clears reconcile_running?
and, on the first round, reaches :normal — Round.run_safe/2 guarantees
one whatever happened.
Functions
@spec await_normal(ProcessHub.hub_id(), non_neg_integer()) :: :ok | {:error, :timeout}
Blocks until the coordinator reaches :normal or timeout_ms elapses.
Returns :ok on reaching :normal (immediately when the hub does not exist
or has no recovery), or {:error, :timeout} otherwise. :normal means the
first reconcile round has completed, so callers SHOULD size the timeout above
reconcile_grace_ms.
@spec complete_first_round( ProcessHub.Hub.t(), ProcessHub.Service.Recovery.Round.result() ) :: ProcessHub.Hub.t()
Completes the first round: moves the coordinator to :normal, dispatches the
transition hook, and fires the async post_recovery_replay.
@spec config_or_disabled(map() | struct()) :: ProcessHub.Hub.recovery_config()
Returns the parsed :auto_recovery config for a settings struct, falling back to
the disabled config for any shape the coordinator would reject.
@spec disabled_config() :: ProcessHub.Hub.recovery_config()
Returns the disabled (default) config.
@spec parse_config(false | true | keyword() | term()) :: {:ok, ProcessHub.Hub.recovery_config()} | {:error, :invalid_auto_recovery | {:invalid_auto_recovery, atom()}}
Parses the :auto_recovery config field into a normalized map.
Accepts the documented shapes:
false— disabled (the default).true— enabled with defaults.keyword()—:reconcile_grace_ms,:reconcile_interval_ms, and:remote_manifest({module, opts}implementingProcessHub.Storage.RemoteManifest, defaultnil).
The superseded keys :marker_path, :replay_timeout_ms,
:recovery_timeout_ms, and :stopped_row_ttl_ms are deprecated: they are
accepted with a WARN and ignored, and will be rejected in a future release.
Returns {:ok, recovery_config}, or {:error, {:invalid_auto_recovery, reason}}
for out-of-range values. Unknown shapes return {:error, :invalid_auto_recovery}
so the caller can decide whether to fall back to disabled or to refuse to start.
@spec prepare_recovery(ProcessHub.hub_id()) :: :ok
Deprecated. Armed the next boot for marker-driven replay by deleting the local marker file.
There is no marker any more: every node reconciles its durable registry against
the cluster continuously, so recovery after an outage needs no pre-boot step.
The function is kept so existing operator tooling keeps running — it logs a
warning and returns :ok without touching the filesystem.
Scheduled for removal in a future release.
@spec prepare_recovery_cluster(ProcessHub.hub_id()) :: {:ok, [node()]} | {:error, :not_alive}
Deprecated. Fanned prepare_recovery/1 out to every hub member.
A no-op for the same reason as prepare_recovery/1; it still reports the hub's
members so existing callers keep matching on {:ok, nodes}. Returns
{:error, :not_alive} when the hub is not running, as before.
Scheduled for removal in a future release.
@spec recovery_state(ProcessHub.hub_id()) :: :recovering | :normal
Returns the coordinator's current :recovery_state.
Returns :normal when the hub does not exist or was started without
:auto_recovery.
@spec round_due?(ProcessHub.Hub.t()) :: boolean()
Returns whether a round triggered by a completed synchronisation round may run.
Rounds are rate-limited to one per reconcile_interval_ms, are never started
before the first (grace-scheduled) round, and never overlap.
@spec schedule_first_round(ProcessHub.Hub.t()) :: ProcessHub.Hub.t()
Schedules the first reconcile round reconcile_grace_ms after coordinator start.
The timer fires whether or not any peer joined, so :normal is reached in
bounded time on every boot. Disabled hubs schedule nothing.
@spec spawn_round(ProcessHub.Hub.t()) :: ProcessHub.Hub.t()
Runs a round in a separate process; replies to the coordinator with
{:reconcile_done, result}. The reply is what clears reconcile_running?
and, on the first round, reaches :normal — Round.run_safe/2 guarantees
one whatever happened.