ClusterHelper.PullEngine (ClusterHelper v1.2.0)

Copy Markdown View Source

Coordinates background pull operations for cluster synchronisation.

Manages async tasks that pull role data from remote nodes, using generation-based change detection to minimise unnecessary RPC traffic. Remote nodes are checked concurrently (bounded by @max_pull_concurrency) so pull cycles stay fast on large clusters.

Summary

Functions

pull_all(server, scope)

Starts background pulls from all connected nodes for the given scope.

pull_all_with_generation_check(server, scope, remote_generations)

Generation-aware periodic pull.

pull_new_node(server, remote_node, local_scopes)

Pulls roles from a newly discovered node, discovering matching scopes first.

Functions

pull_all(server, scope)

@spec pull_all(pid(), atom()) :: :ok

Starts background pulls from all connected nodes for the given scope.

Each pull runs in a separate Task via Task.Supervisor, and results are sent back to server as {:pull_update_node, scope, node, roles, gen}.

pull_all_with_generation_check(server, scope, remote_generations)

@spec pull_all_with_generation_check(pid(), atom(), %{optional(atom()) => map()}) ::
  :ok

Generation-aware periodic pull.

For each live node, checks the remote generation first (concurrently across nodes). Only does a full role pull when the generation has changed. Also detects and reports stale nodes (in ETS but no longer in Node.list()).

Results are sent back as:

  • {:pull_update_node, scope, node, roles, gen} — gen is an integer when the remote generation was known, nil otherwise
  • {:stale_node, scope, node} — node no longer in cluster

pull_new_node(server, remote_node, local_scopes)

@spec pull_new_node(pid(), node(), MapSet.t(atom())) :: :ok

Pulls roles from a newly discovered node, discovering matching scopes first.