raft_fleet v0.6.1 RaftFleet View Source

Public interface functions of raft_fleet.

See also RaftFleet.Config for available application configs.

Link to this section Summary

Functions

Activates Node.self()

Queries the current nodes which have been activated using activate/1 in the cluster

Registers a new consensus group identified by name

Queries already registered consensus groups

Deactivates Node.self()

Inspects members of consensus groups and finds a group (if any) in which no leader exists

Executes a read-only query on the replicated value identified by name

Removes an existing consensus group identified by name

Removes member pids that reside in the specified dead node from all existing consensus groups

Gets information about nodes that are active but unreachable from Node.self()

Tries to find the current leader of the consensus group specified by name

Link to this section Functions

Link to this function activate(zone) View Source
activate(RaftFleet.ZoneId.t()) :: :ok | {:error, :not_inactive}

Activates Node.self().

When :raft_fleet is started as an OTP application, the node is not active; to host consensus group members each node must be explicitly activated. zone is an ID of data center zone which this node belongs to. zone is used to determine which nodes to replicate data: RaftFleet tries to place members of each consensus group across multiple zones for maximum availability.

Node activation by calling this function should be done after the node is fully connected to the other existing nodes; otherwise there is a possibility (although it is small) that the cluster forms partitioned subset of active nodes.

Link to this function active_nodes() View Source
active_nodes() :: %{optional(RaftFleet.ZoneId.t()) => [node()]}

Queries the current nodes which have been activated using activate/1 in the cluster.

This function sends a query to a leader of the RaftFleet.Cluster consensus group, which is managed internally by raft_fleet. The returned value is grouped by zone IDs which have been passed to activate/1. This function exits if no active node exists in the cluster.

Link to this function add_consensus_group(name, n_replica, rv_config, await_timeout \\ 5000) View Source
add_consensus_group(
  atom(),
  pos_integer(),
  RaftedValue.Config.t(),
  pos_integer()
) :: :ok | {:error, :already_added | :no_leader | any()}

Registers a new consensus group identified by name.

name is used as a registered name for member processes of the new consensus group. n_replica is the number of replicas (Raft member processes implemented as RaftedValue.Server). If you want all active nodes to host member processes, specify sufficiently large integer as n_replica. For explanation of rv_config see RaftedValue.make_config/2.

If you configure raft_fleet to persist Raft logs & snapshots (see :persistence_dir_parent in RaftFleet.Config) and the consensus group with name had been removed by remove_consensus_group/1, then add_consensus_group/3 will restore the state of the consensus group from the snapshot and log files.

The caller is blocked until the newly spawned leader becomes ready. await_timeout specifies how many milliseconds to wait for the initialization.

Link to this function command(name, command_arg, timeout \\ 500, retry \\ 3, retry_interval \\ 1000, call_module \\ :gen_statem) View Source
command(
  atom(),
  RaftedValue.Data.command_arg(),
  pos_integer(),
  non_neg_integer(),
  pos_integer(),
  module()
) :: {:ok, RaftedValue.Data.command_ret()} | {:error, :no_leader}

Executes a command on the replicated value identified by name.

The target consensus group identified by name must be registered beforehand using add_consensus_group/3. This function automatically resolves the leader process of the consensus group, caches PID of the current leader in local ETS table and send the given command to the leader.

timeout is used in each synchronous messaging. In order to tolerate temporal absences of leaders during Raft leader elections, it retries requests up to retry. Before retrying requests this function sleeps for retry_interval milliseconds. Thus for worst case this function blocks the caller for timeout * (retry + 1) + retry_interval * retry. Note that for complete masking of leader elections retry_interval * retry must be sufficiently longer than the time scale for leader elections (:election_timeout in RaftedValue.Config.t).

call_module can be changed from the default (:gen_statem module) with an alternative module that exports call/3 for optimization of remote message passing.

See also RaftedValue.command/5.

Link to this function consensus_groups() View Source
consensus_groups() :: %{optional(atom()) => pos_integer()}

Queries already registered consensus groups.

This function sends a query to a leader of the RaftFleet.Cluster consensus group, which is managed internally by raft_fleet. The returned value is a map whose keys and values are consensus group name and number of replicas of the group. This function exits if no active node exists in the cluster.

Link to this function deactivate() View Source
deactivate() :: :ok | {:error, :inactive}

Deactivates Node.self().

Call this function before you remove an ErlangVM from your cluster. Note that calling this function does not immediately remove consensus member processes in this node; these processes will be gradually migrated to other nodes by periodic rebalancing.

Link to this function find_consensus_group_with_no_established_leader() View Source
find_consensus_group_with_no_established_leader() ::
  :ok | {group_name :: atom(), [{node(), map()}]}

Inspects members of consensus groups and finds a group (if any) in which no leader exists.

If one found, returns the name of the consensus group and also statuses (as maps) of existing members. If no consensus group is in trouble, returns :ok.

Target consensus groups are:

This function is primarily intended to be used within remote console. Use this function to detect problematic consensus group in your cluster.

Link to this function query(name, query_arg, timeout \\ 500, retry \\ 3, retry_interval \\ 1000, call_module \\ :gen_statem) View Source
query(
  atom(),
  RaftedValue.Data.query_arg(),
  pos_integer(),
  non_neg_integer(),
  pos_integer(),
  module()
) :: {:ok, RaftedValue.Data.query_ret()} | {:error, :no_leader}

Executes a read-only query on the replicated value identified by name.

See command/6 for explanations of name, timeout, retry, retry_interval and call_module. See also RaftedValue.query/4.

Link to this function remove_consensus_group(name) View Source
remove_consensus_group(atom()) :: :ok | {:error, :not_found | :no_leader}

Removes an existing consensus group identified by name.

Removing a consensus group will eventually trigger terminations of all members of the group. The replicated value held by the group will be discarded.

Note that remove_consensus_group/1 does not immediately terminate existing member processes; they will be terminated afterward by background worker process (see also :balancing_interval in RaftFleet.Config). Note also that, if Raft logs and snapshots has been created (see :persistence_dir_parent in RaftFleet.Config), remove_consensus_group/1 does not remove these files.

Link to this function remove_dead_pids_located_in_dead_node(dead_node) View Source
remove_dead_pids_located_in_dead_node(node()) :: :ok

Removes member pids that reside in the specified dead node from all existing consensus groups.

Target consensus groups are:

If a target consensus group does not have an established leader, then this function tries to remove dead pids (if any) by using RaftedValue.force_remove_member/2.

This function crashes if the RaftFleet.Cluster consensus group does not have a leader. Each of the effects of this function is idempotent; you can freely call this function multiple times in case of failure.

Use this function to resolve issues when e.g. some node suddenly died without cleaning up itself. The caller must be sure that the dead_node has definitely died.

Link to this function unreachable_nodes() View Source
unreachable_nodes() :: %{optional(node()) => unreachable_since}
when unreachable_since: pos_integer()

Gets information about nodes that are active but unreachable from Node.self().

RaftFleet periodically checks reachability to the other active nodes. This function retrieves results of the checks as a map. Each key of the returned map is a currently unreachable node (i.e. if there’s no problem %{} is returned), whereas each value is a timestamp (seconds since epoch) at which the node was recognized as unreachable.

Note that returned timestamps are not so accurate about when the node failed, since they are obtained by healthchecks with an interval of :node_purge_reconnect_interval (see also RaftFleet.Config). Note also that, when a node remains unreachable for more than :node_purge_failure_time_window, the node will be automatically purged from the active nodes. After purging the failed node will not be included in return value of this function (as the node is no longer active).

Link to this function whereis_leader(name) View Source
whereis_leader(atom()) :: nil | pid()

Tries to find the current leader of the consensus group specified by name.

Usually you don’t have to use this function as command/5 and query/5 automatically resolves where the leader resides. This function is useful when you want to inspect status of a consensus group by using e.g. RaftedValue.status/1.