raft_kv v0.2.0 RaftKV.SplitMergePolicy View Source

An Elixir struct to specify when to split/merge shards in a keyspace.

:raft_kv uses the following 3 stats to judge whether splitting/merging should be done:

  1. number of keys in a shard
  2. aggregated data size (in an arbitrary unit) in a shard
  3. current load (computing resource required, in an arbitrary unit) of a shard

Fields:

  • :max_shards Maximum number of shards. If number of shards is the same as this value, no further split occurs.
  • :min_shards Minimum number of shards. If number of shards is the same as this value, no further merge occurs.
  • :max_keys_per_shard Threshold number of keys for shard split. Shards that contains more keys than this value become candidates for split. If nil, shards are not split due to number of keys. Defaults to nil.
  • :max_size_per_shard Threshold size for shard split. Shards whose aggregated size exceeds this value become candidates for split. If nil, shards are not split due to size. Defaults to nil.
  • :max_load_per_shard Threshold load for shard split. Shards that have been experiencing load above this value become candidates for split. If nil, shards are not split due to load. Defaults to nil.
  • :load_per_query_to_missing_key RaftKV.query/4 returns {:error, :key_not_found} if target key does not exist. This value is used as the “load” for such queries. Defaults to 0.
  • :merge_threshold_ratio: For each of the 3 types of split thresholds above (:max_keys_per_shard, :max_size_per_shard and :max_load_per_shard), merge thresholds are calculated by multiplying this ratio. Consecutive 2 shards become candidates for merge if they together (1) contain less keys than the threshold, (2) contain smaller size than the threshold, and (3) experience less load than the threshold. Defaults to 0.5.

Link to this section Summary

Functions

Type-aware getter for load_per_query_to_missing_key

Type-aware setter for load_per_query_to_missing_key

Type-aware getter for max_keys_per_shard

Type-aware setter for max_keys_per_shard

Type-aware getter for max_load_per_shard

Type-aware setter for max_load_per_shard

Type-aware getter for max_shards

Type-aware setter for max_shards

Type-aware getter for max_size_per_shard

Type-aware setter for max_size_per_shard

Type-aware getter for merge_threshold_ratio

Type-aware setter for merge_threshold_ratio

Type-aware getter for min_shards

Type-aware setter for min_shards

Creates a new instance of RaftKV.SplitMergePolicy by using the given dict

A variant of new/1 which returns t or raise if validation fails

Updates an existing instance of RaftKV.SplitMergePolicy with the given dict. The values in the dict are validated by each field’s valid?/1 function. Returns {:ok, valid_struct} or {:error, reason}

A variant of update/2 which returns t or raise if validation fails. In other words, update/2 followed by Croma.Result.get!/1

Checks if the given value belongs to t/0 or not

Link to this section Types

Link to this type t() View Source
t() :: %RaftKV.SplitMergePolicy{
  load_per_query_to_missing_key: Croma.NonNegInteger.t(),
  max_keys_per_shard: Croma.TypeGen.Nilable.Croma.PosInteger.t(),
  max_load_per_shard: Croma.TypeGen.Nilable.Croma.PosInteger.t(),
  max_shards: Croma.PosInteger.t(),
  max_size_per_shard: Croma.TypeGen.Nilable.Croma.PosInteger.t(),
  merge_threshold_ratio: RaftKV.SplitMergePolicy.MergeThresholdRatio.t(),
  min_shards: Croma.PosInteger.t()
}

Link to this section Functions

Link to this function load_per_query_to_missing_key(split_merge_policy) View Source
load_per_query_to_missing_key(t()) :: Croma.NonNegInteger.t()

Type-aware getter for load_per_query_to_missing_key.

Link to this function load_per_query_to_missing_key(s, field) View Source
load_per_query_to_missing_key(t(), Croma.NonNegInteger.t()) :: t()

Type-aware setter for load_per_query_to_missing_key.

Link to this function max_keys_per_shard(split_merge_policy) View Source
max_keys_per_shard(t()) :: Croma.TypeGen.Nilable.Croma.PosInteger.t()

Type-aware getter for max_keys_per_shard.

Link to this function max_keys_per_shard(s, field) View Source
max_keys_per_shard(t(), Croma.TypeGen.Nilable.Croma.PosInteger.t()) :: t()

Type-aware setter for max_keys_per_shard.

Link to this function max_load_per_shard(split_merge_policy) View Source
max_load_per_shard(t()) :: Croma.TypeGen.Nilable.Croma.PosInteger.t()

Type-aware getter for max_load_per_shard.

Link to this function max_load_per_shard(s, field) View Source
max_load_per_shard(t(), Croma.TypeGen.Nilable.Croma.PosInteger.t()) :: t()

Type-aware setter for max_load_per_shard.

Link to this function max_shards(split_merge_policy) View Source
max_shards(t()) :: Croma.PosInteger.t()

Type-aware getter for max_shards.

Link to this function max_shards(s, field) View Source
max_shards(t(), Croma.PosInteger.t()) :: t()

Type-aware setter for max_shards.

Link to this function max_size_per_shard(split_merge_policy) View Source
max_size_per_shard(t()) :: Croma.TypeGen.Nilable.Croma.PosInteger.t()

Type-aware getter for max_size_per_shard.

Link to this function max_size_per_shard(s, field) View Source
max_size_per_shard(t(), Croma.TypeGen.Nilable.Croma.PosInteger.t()) :: t()

Type-aware setter for max_size_per_shard.

Link to this function merge_threshold_ratio(split_merge_policy) View Source
merge_threshold_ratio(t()) :: RaftKV.SplitMergePolicy.MergeThresholdRatio.t()

Type-aware getter for merge_threshold_ratio.

Link to this function merge_threshold_ratio(s, field) View Source
merge_threshold_ratio(t(), RaftKV.SplitMergePolicy.MergeThresholdRatio.t()) ::
  t()

Type-aware setter for merge_threshold_ratio.

Link to this function min_shards(split_merge_policy) View Source
min_shards(t()) :: Croma.PosInteger.t()

Type-aware getter for min_shards.

Link to this function min_shards(s, field) View Source
min_shards(t(), Croma.PosInteger.t()) :: t()

Type-aware setter for min_shards.

Creates a new instance of RaftKV.SplitMergePolicy by using the given dict.

For missing fields, default/0 of each field type will be used.

Returns {:ok, valid_struct} or {:error, reason}. The values in the dict are validated by each field’s valid?/1 function.

A variant of new/1 which returns t or raise if validation fails.

In other words, new/1 followed by Croma.Result.get!/1.

Updates an existing instance of RaftKV.SplitMergePolicy with the given dict. The values in the dict are validated by each field’s valid?/1 function. Returns {:ok, valid_struct} or {:error, reason}.

Link to this function update!(s, dict) View Source
update!(t(), Dict.t()) :: t()

A variant of update/2 which returns t or raise if validation fails. In other words, update/2 followed by Croma.Result.get!/1.

Checks if the given value belongs to t/0 or not.