View Source ProcessHub.Service.Recovery (ProcessHub v0.7.0)

Orphan reconcile: every round converges the cluster toward the hub's declared list (ProcessHub.Service.DeclaredChildren) — it starts declared − observed running anywhere through the normal start path with check_existing: true, and stops running children whose declared entry was removed. The same difference covers a whole-cluster restart and a single-node rejoin; stop knowledge is list absence and never expires.

Experimental

The orphan reconcile (the :auto_recovery lifecycle) is experimental and may change in future releases. Use in production at your own discretion.

The first round runs reconcile_grace_ms after coordinator start, later rounds follow completed synchronisation rounds, rate-limited to one per reconcile_interval_ms. This module owns the :auto_recovery config, the scheduling, and the recovery lifecycle; the round itself lives in ProcessHub.Service.Recovery.Round and the coordinator stays the GenServer. See guides/Persistence.md for the model.

Summary

Functions

Blocks until the coordinator reaches :normal or timeout_ms elapses.

Completes the first round: moves the coordinator to :normal, dispatches the transition hook, and fires the async post_recovery_replay.

Returns the parsed :auto_recovery config for a settings struct, falling back to the disabled config for any shape the coordinator would reject.

Returns the disabled (default) config.

Parses the :auto_recovery config field into a normalized map.

Deprecated. Armed the next boot for marker-driven replay by deleting the local marker file.

Deprecated. Fanned prepare_recovery/1 out to every hub member.

Returns the coordinator's current :recovery_state.

Returns whether a round triggered by a completed synchronisation round may run.

Schedules the first reconcile round reconcile_grace_ms after coordinator start.

Runs a round in a separate process; replies to the coordinator with {:reconcile_done, result}. The reply is what clears reconcile_running? and, on the first round, reaches :normalRound.run_safe/2 guarantees one whatever happened.

Functions

Link to this function

await_normal(hub_id, timeout_ms \\ 60000)

View Source
@spec await_normal(ProcessHub.hub_id(), non_neg_integer()) :: :ok | {:error, :timeout}

Blocks until the coordinator reaches :normal or timeout_ms elapses.

Returns :ok on reaching :normal (immediately when the hub does not exist or has no recovery), or {:error, :timeout} otherwise. :normal means the first reconcile round has completed, so callers SHOULD size the timeout above reconcile_grace_ms.

Link to this function

complete_first_round(hub, result)

View Source

Completes the first round: moves the coordinator to :normal, dispatches the transition hook, and fires the async post_recovery_replay.

Link to this function

config_or_disabled(hub_conf)

View Source
@spec config_or_disabled(map() | struct()) :: ProcessHub.Hub.recovery_config()

Returns the parsed :auto_recovery config for a settings struct, falling back to the disabled config for any shape the coordinator would reject.

@spec disabled_config() :: ProcessHub.Hub.recovery_config()

Returns the disabled (default) config.

@spec parse_config(false | true | keyword() | term()) ::
  {:ok, ProcessHub.Hub.recovery_config()}
  | {:error, :invalid_auto_recovery | {:invalid_auto_recovery, atom()}}

Parses the :auto_recovery config field into a normalized map.

Accepts the documented shapes:

  • false — disabled (the default).
  • true — enabled with defaults.
  • keyword():reconcile_grace_ms, :reconcile_interval_ms, and :remote_manifest ({module, opts} implementing ProcessHub.Storage.RemoteManifest, default nil).

The superseded keys :marker_path, :replay_timeout_ms, :recovery_timeout_ms, and :stopped_row_ttl_ms are deprecated: they are accepted with a WARN and ignored, and will be rejected in a future release.

Returns {:ok, recovery_config}, or {:error, {:invalid_auto_recovery, reason}} for out-of-range values. Unknown shapes return {:error, :invalid_auto_recovery} so the caller can decide whether to fall back to disabled or to refuse to start.

Link to this function

prepare_recovery(hub_id)

View Source
This function is deprecated. The recovery marker is no longer used; this is a no-op. See migration-guide.md.
@spec prepare_recovery(ProcessHub.hub_id()) :: :ok

Deprecated. Armed the next boot for marker-driven replay by deleting the local marker file.

There is no marker any more: every node reconciles its durable registry against the cluster continuously, so recovery after an outage needs no pre-boot step. The function is kept so existing operator tooling keeps running — it logs a warning and returns :ok without touching the filesystem.

Scheduled for removal in a future release.

Link to this function

prepare_recovery_cluster(hub_id)

View Source
This function is deprecated. The recovery marker is no longer used; this is a no-op. See migration-guide.md.
@spec prepare_recovery_cluster(ProcessHub.hub_id()) ::
  {:ok, [node()]} | {:error, :not_alive}

Deprecated. Fanned prepare_recovery/1 out to every hub member.

A no-op for the same reason as prepare_recovery/1; it still reports the hub's members so existing callers keep matching on {:ok, nodes}. Returns {:error, :not_alive} when the hub is not running, as before.

Scheduled for removal in a future release.

@spec recovery_state(ProcessHub.hub_id()) :: :recovering | :normal

Returns the coordinator's current :recovery_state.

Returns :normal when the hub does not exist or was started without :auto_recovery.

@spec round_due?(ProcessHub.Hub.t()) :: boolean()

Returns whether a round triggered by a completed synchronisation round may run.

Rounds are rate-limited to one per reconcile_interval_ms, are never started before the first (grace-scheduled) round, and never overlap.

Link to this function

schedule_first_round(hub)

View Source
@spec schedule_first_round(ProcessHub.Hub.t()) :: ProcessHub.Hub.t()

Schedules the first reconcile round reconcile_grace_ms after coordinator start.

The timer fires whether or not any peer joined, so :normal is reached in bounded time on every boot. Disabled hubs schedule nothing.

@spec spawn_round(ProcessHub.Hub.t()) :: ProcessHub.Hub.t()

Runs a round in a separate process; replies to the coordinator with {:reconcile_done, result}. The reply is what clears reconcile_running? and, on the first round, reaches :normalRound.run_safe/2 guarantees one whatever happened.