Load Balancer Reference

Complete API reference for all public modules in rpc_load_balancer. Module-level docs (h RpcLoadBalancer in IEx) carry the same information; this page collects it in one place.

RpcLoadBalancer

Top-level module. It is three things at once:

• the public API for RPC calls/casts, node selection, and random-node helpers
• a per-instance Supervisor (start_link/1) for one load balancer
• a use macro that generates a named load balancer module

Types

@type name :: atom()

A load balancer name is any atom, including a module name.

use RpcLoadBalancer

defmodule MyApp.LoadBalancer do
  use RpcLoadBalancer,
    selection_algorithm: RpcLoadBalancer.LoadBalancer.SelectionAlgorithm.HashRing,
    node_match_list: ["my_app"]
end

Accepts every start_link/1 option except :name, which is set to the using module. Defines:

Functions

start_link(opts)

Starts a Supervisor for a single balancer instance. Its children are the selection algorithm's child_specs/2 (if any) followed by the RpcLoadBalancer.LoadBalancer GenServer, under :one_for_all. Returns once the balancer has joined its :pg group.

Options:

• :name (required) — registered name for the balancer; also used as the :pg group name
• :selection_algorithm — module implementing SelectionAlgorithm (default: SelectionAlgorithm.Random)
• :algorithm_opts — keyword list forwarded to the algorithm's init/2 and child_specs/2 callbacks (default: [])
• :node_match_list — controls whether this node joins the :pg group (default: :all)
  • :all — every node joins

  • [String.t() | Regex.t()] — joins only if RpcLoadBalancer.NodeFilter.matches?/2 is true for at least one entry

• :drain_timeout — max milliseconds to wait for in-flight calls during shutdown (default: 15_000; :infinity allowed)

Returns: Supervisor.on_start()

The GenServer is registered as :"#{name}_server"; the supervisor as name.

get_members(load_balancer_name)

Returns the deduplicated list of nodes with a live member process in the balancer's :pg group.

Returns:

• {:ok, [node()]} when members exist
• {:error, %ErrorMessage{code: :service_unavailable}} when the group is empty

select_node(load_balancer_name, opts \\ [])

Selects a node from the balancer's registered members using the configured algorithm. Does not make an RPC. Emits [:rpc_load_balancer, :node_selected].

Options: all forwarded verbatim to the algorithm's choose_from_nodes/3 (e.g. key: "user:123" for HashRing)

Returns:

• {:ok, node()} on success
• {:error, %ErrorMessage{code: :service_unavailable}} when no nodes are registered

For connection-tracking algorithms the caller must call SelectionAlgorithm.release_node/3 after the work completes — see Connection Tracking.

call(node, module, fun, args, opts \\ [])

Synchronous RPC. Wrapped in the [:rpc_load_balancer, :rpc] telemetry span.

• Without :load_balancer — :erpc.call/5 to node.
• With :load_balancer — node is ignored. If call_directly? (option or config) is true, or the algorithm's local?/0 is true, runs apply(module, fun, args) locally. Otherwise selects a member, tracks the call for draining, runs :erpc.call/5, then calls the algorithm's release_node/2. If no member is registered, retries per the retry options before failing.

Options:

• :timeout — call timeout in milliseconds (default: 10_000)
• :load_balancer — name of a running load balancer to route through
• :key — forwarded to the selection algorithm (used by HashRing)
• :call_directly? — execute locally via apply/3 (default: RpcLoadBalancer.Config.call_directly?/0)
• :retry?, :retry_count, :retry_sleep — no-route retry, see RpcLoadBalancer.Retry below

Returns:

• {:ok, result} on success
• {:error, %ErrorMessage{code: :request_timeout}} on :erpc timeout
• {:error, %ErrorMessage{code: :service_unavailable}} on :noconnection, other :erpc failures, or no members after retries
• {:error, %ErrorMessage{code: :bad_request}} on :erpc :badarg

cast(node, module, fun, args, opts \\ [])

Asynchronous fire-and-forget RPC. Same routing rules as call/5, using :erpc.cast/4 remotely and spawn/3 locally.

Options: :load_balancer, :key, :call_directly?, :retry?, :retry_count, :retry_sleep

Returns:

• :ok when dispatched
• {:error, %ErrorMessage{}} on failure to dispatch or no members after retries

call_on_random_node(node_filter, module, fun, args, opts \\ [])

Picks a random node from Node.list/0 whose name matches node_filter (via RpcLoadBalancer.NodeFilter.matches?/2 — substring or regex, honouring excluded_node_patterns), then runs call/5 on it. If the current node matches the filter or :call_directly? is true, executes locally via apply/3.

Retries when no node matches, per the retry options.

Options:

• :timeout — call timeout in milliseconds
• :load_balancer — enrols the call in that balancer's connection draining (no effect on selection). Requires a balancer instance with that name to be running on the calling node
• :call_directly? — execute locally (default: from config)
• :retry?, :retry_count, :retry_sleep — see RpcLoadBalancer.Retry

Returns:

• {:ok, result} on success
• {:error, %ErrorMessage{code: :service_unavailable, message: "no nodes in cluster found with that filter"}} when nothing matches after retries
• any call/5 error from the selected node

cast_on_random_node(node_filter, module, fun, args, opts \\ [])

Same as call_on_random_node/5 but uses cast/5 (or spawn/3 locally).

Returns: :ok or {:error, %ErrorMessage{}}

RpcLoadBalancer.Config

Reads application config with defaults. Override in config/*.exs:

config :rpc_load_balancer,
  call_directly?: false,
  retry?: true,
  retry_count: 5,
  excluded_node_patterns: []

RpcLoadBalancer.Retry

with_retry(opts \\ [], fun)

Calls fun until it returns something other than :retry, sleeping :retry_sleep between attempts. Returns the first non-:retry result, or :error once retries are exhausted.

Options:

• :retry? — enable retrying (default: RpcLoadBalancer.Config.retry?/0)
• :retry_count — retries after the first attempt; non_neg_integer() or :infinity (default: RpcLoadBalancer.Config.retry_count/0)
• :retry_sleep — milliseconds between attempts (default: 5_000)

Used by call/5/cast/5 for empty :pg groups and by the random-node helpers for empty filter results. Never retries a dispatched RPC.

RpcLoadBalancer.NodeFilter

matches?(node_name, filter, excluded_patterns \\ Config.excluded_node_patterns())

Returns true when to_string(node_name) =~ filter and no excluded pattern is present in the node's short name (before @) but absent from the filter. filter may be a string (substring match) or a Regex; regex filters are checked for excluded patterns via Regex.source/1.

Used for :node_match_list membership and the node_filter argument of the random-node helpers.

RpcLoadBalancer.Metrics

metrics()

Returns [Telemetry.Metrics.t()] ready for any reporter:

Full event/metadata reference: Telemetry and Metrics.

RpcLoadBalancer.LoadBalancer

GenServer started by RpcLoadBalancer.start_link/1. On init/1 it seeds the drainer and counter index registries, records the algorithm in AlgorithmCache, runs the algorithm's init/2, joins the :pg group if node_match_list matches, and subscribes to :pg membership changes (:pg.monitor/2, OTP 25.1+). Join/leave notifications are forwarded to the algorithm's on_node_change/2. On terminate/2 it leaves the group and blocks in Drainer.drain/2 for up to :drain_timeout.

You don't call this module directly.

RpcLoadBalancer.LoadBalancer.SelectionAlgorithm

Behaviour plus dispatch layer. The dispatch functions (init/3, choose_from_nodes/4, choose_nodes/5, on_node_change/3, release_node/3, local?/1, child_specs/3, caches/1) take the algorithm module first, check function_exported?/3 for optional callbacks, and supply defaults.

Callbacks

Required

@callback choose_from_nodes(load_balancer_name(), [node()], opts :: keyword()) :: node()

Pick one node from the member list. Called concurrently from every calling process. Options come from select_node/2 (all of them) or call/5/cast/5 (:key, :call_directly?, :load_balancer only).

Optional

@callback init(load_balancer_name(), opts :: keyword()) :: :ok

Once at balancer startup, before joining :pg. Receives algorithm_opts.

@callback choose_nodes(load_balancer_name(), [node()], pos_integer(), opts :: keyword()) :: [node()]

Pick count distinct nodes (replicas). Default: Enum.shuffle/1 |> Enum.take(count).

@callback on_node_change(load_balancer_name(), {:joined | :left, [node()]}) :: :ok

:pg membership changed. Default: no-op.

@callback release_node(load_balancer_name(), node()) :: :ok

A load-balanced call/5/cast/5 finished on node. Default: no-op.

@callback local?() :: boolean()

true → skip selection and :erpc; run apply/3 / spawn/3 locally. Default: false.

@callback child_specs(load_balancer_name(), opts :: keyword()) :: [Supervisor.child_spec()]

Children started under the balancer supervisor before the LoadBalancer GenServer. Default: [].

@callback caches() :: [module()]

elixir_cache modules the algorithm needs. The application supervisor starts them at boot for the built-in algorithms only. Default: [].

Dispatch helpers you may call

• get_algorithm(name) — {:ok, module | nil}; the algorithm recorded for a running balancer

• choose_from_nodes(algorithm, name, nodes, opts \\ []) — runs the callback and emits [:rpc_load_balancer, :node_selected] (or [..., :empty] for [], then lets the algorithm raise)
• choose_nodes(algorithm, name, nodes, count, opts \\ [])
• release_node(algorithm, name, node)

Built-in Algorithms

All live under RpcLoadBalancer.LoadBalancer.SelectionAlgorithm.*. Costs from bench/README.md (8 nodes, single process, M1 Max).

Random (default)

Enum.random/1. Stateless. ~0.11 μs.

Implements: choose_from_nodes/3

RoundRobin

Cycles through the member list with a shared atomic counter (CounterCache slot 1). Auto-resets above 10,000,000. ~0.77 μs, constant.

Implements: init/2, choose_from_nodes/3

WeightedRoundRobin

Round robin over an expanded list with each node repeated weight times. Weights stored in :persistent_term at init/2; the expanded list is cached in WeightedRoundRobinCache (ETS) and rebuilt after membership changes. Missing nodes default to weight 1. ~0.88 μs, constant.

Algorithm options: weights: %{node() => pos_integer()} (default %{})

Implements: init/2, choose_from_nodes/3, on_node_change/2, caches/0

LeastConnections

Per-node :counters in CounterCache. Single-pass scan for the lowest count, increment on select, decrement on release_node/2, reset on leave. ~1.18 μs, linear in members.

Implements: init/2, choose_from_nodes/3, on_node_change/2, release_node/2

PowerOfTwo

Two random members, pick the one with fewer connections. Same counters as LeastConnections. ~1.32 μs, two reads regardless of size.

Implements: init/2, choose_from_nodes/3, on_node_change/2, release_node/2

HashRing

Consistent hashing via libring (:erlang.phash2/2 over a 2^32 range, weight virtual nodes per member). Routes by the :key option; falls back to random without one. choose_nodes/4 returns N distinct nodes for a key. Weight in :persistent_term; the built ring cached in HashRingCache (ETS), invalidated on membership change and rebuilt lazily. ~2.02 μs, near-constant.

Algorithm options: weight: pos_integer() (default 128)

Implements: init/2, choose_from_nodes/3, choose_nodes/4, on_node_change/2, caches/0

LeastCpu

Routes to the member with the lowest cached CPU, picking randomly among nodes within cpu_threshold of the minimum. A per-balancer LeastCpu.Poller GenServer samples local CPU (:cpu_sup.util/0) and fetches remote members' readings via :erpc.multicall/5 every poll_interval, writing to the node-keyed NodeCpuCache. Missing/stale entries read as 50%. ~12 μs, linear.

Algorithm options:

• :poll_interval (default 5_000)
• :poll_startup_jitter (default 60_000)
• :cpu_cache_ttl (default 10_000)
• :cpu_threshold (default 5.0)
• :cpu_sampler (default &:cpu_sup.util/0)

Telemetry: [:rpc_load_balancer, :least_cpu, :poll, :start | :stop | :exception | :remote_error].

Implements: init/2, choose_from_nodes/3, child_specs/2, caches/0

CallDirect

local?/0 returns true, so load-balanced calls run apply/3 and casts spawn/3 on the local node. Nothing is selected, no :pg lookup. ~0.04 μs. For tests and single-node deployments.

Implements: local?/0, choose_from_nodes/3 (returns node(); unused on the RPC path)

RpcLoadBalancer.LoadBalancer.Drainer

Tracks in-flight load-balanced calls per balancer with an atomic counter (DrainerCache) so shutdown can wait for them.

• register(load_balancer_name) — returns the counter index for a balancer (allocating on first use)
• track_call(index) / release_call(index) — increment / decrement
• in_flight_count(index) — current count
• drain(index, timeout \\ 15_000) — poll every 50 ms until the count reaches 0; :ok or {:error, :timeout}

call/5, cast/5, and the random-node helpers (when given :load_balancer) wrap the RPC in track_call/release_call (in an after block). LoadBalancer.terminate/2 calls drain/2 after leaving :pg.

Internal Modules

Not part of the public API; documented for contributors.