RaftFleet defines the following application configs:
:balancing_interval- Time interval between periodic triggers of workers whose job is to re-balance Raft member processes across the cluster.
- The actual value used is obtained by
Application.get_env(:raft_fleet, :balancing_interval, 60000), i.e. it defaults to 1 minute.
:leader_pid_cache_refresh_interval- Interval time in milliseconds of leader pids cached in each nodes' local ETS tables.
- The actual value used is obtained by
Application.get_env(:raft_fleet, :leader_pid_cache_refresh_interval, 300000), i.e. it defaults to 5 minutes.
:follower_addition_delay- Time duration in milliseconds to wait for before spawning a new follower for a consensus group. Concurrently spawning multiple followers may lead to race conditions (adding a member can only be done one-by-one). Although this race condition can be automatically resolved by retries and thus is basically harmless, this configuration item may be useful to reduce useless error logs.
- The actual value used is obtained by
Application.get_env(:raft_fleet, :follower_addition_delay, 200).
:node_purge_failure_time_window- A node is considered "unhealthy" if it has been disconnected from the other nodes
without declaring itself as
inactive(by callingRaftFleet.deactivate/0). RaftFleet tries to reconnect to unhealthy nodes in order to recover from short-term issues such as temporary network failures (see also:node_purge_reconnect_intervalbelow). To handle longer-term issues, RaftFleet automatically removes nodes that remain "unhealthy" for this time window (in milliseconds) from the list of participating active nodes. After removal, consensus member processes are automatically re-balanced within remaining active nodes. - The actual value used is obtained by
Application.get_env(:raft_fleet, :node_purge_failure_time_window, 600000), i.e. it defaults to 10 minutes.
- A node is considered "unhealthy" if it has been disconnected from the other nodes
without declaring itself as
:node_purge_reconnect_interval- Time interval (in milliseconds) of periodic reconnect attempts to disconnected nodes.
- The actual value used is obtained by
Application.get_env(:raft_fleet, :node_purge_reconnect_interval, 60000), i.e. it defaults to 1 minute.
:rafted_value_config_maker- A module that implements
RaftFleet.RaftedValueConfigMakerbehaviour. The module is used when:raft_fleetneeds to construct aRaftedValue.Config.t/0. To be more precise, it's used- in
RaftFleet.add_consensus_group/1, or - when restoring
RaftFleet.Clusterand the other consensus groups from log & snapshot files.
- in
- We recommend that you specify your own callback module as
:rafted_value_config_maker, since, if omitted, you have the following limitations:RaftFleet.add_consensus_group/1cannot be used (you must useRaftFleet.add_consensus_group/3instead), and- when restoring from log & snapshot files, some consensus groups may not be restored (this limitation comes from the current implementation, but we don't have plan to fix this issue).
- Note that you can customize the
RaftedValue.Config.t/0used by theRaftFleet.Clusterconsensus group. You can useRaftFleet.Cluster.make_rv_config/1in your callback implementation. - Note also that libraries using
:raft_fleetmay add their own consensus groups (just as in the same way asRaftFleet.Clusterexpained above). Implementations ofRaftFleet.RaftedValueConfigMakerbehaviour must be aware of such consensus groups and return appropriate configuration whenRaftFleet.RaftedValueConfigMaker.make/1is called.
- A module that implements
:per_member_options_maker- A module that implements
RaftFleet.PerMemberOptionsMakerbehaviour. The module is used when constructing a 2nd argument ofRaftedValue.start_link/2, i.e., when starting a new consensus member process. This configuration provides a way to customize options (such as whether to persist Raft logs & snapshots) for each consensus group member. - Defaults to
nil, which means that the default options ofRaftedValue.start_link/2are used for all consensus groups. - Note that
RaftFleet.Cluster(a special consensus group to manage metadata for:raft_fleet) also uses:per_member_options_makermodule (if set). Callback implementation must handleRaftFleet.Clusterappropriately, in addition to consensus group names that are explicitly added byRaftFleet.add_consensus_group/3. - Note also that you cannot specify
:nameoption by your callback implementation as it's fixed by:raft_fleet.
- A module that implements
Note that each raft_fleet process uses application configs stored in the local node. If you want to configure the options above you must set them on all nodes in your cluster.
Summary
Functions
@spec balancing_interval() :: pos_integer()
@spec follower_addition_delay() :: pos_integer()
@spec leader_pid_cache_refresh_interval() :: pos_integer()
@spec node_purge_failure_time_window() :: pos_integer()
@spec node_purge_reconnect_interval() :: pos_integer()
@spec per_member_options_maker() :: nil | module()
@spec rafted_value_config_maker() :: nil | module()