Behaviour for enforcing rate limits across Legion agents.

A rate limiter decides whether an agent may proceed under a list of Legion.RateLimiter.Rules. Each rule pairs an identity - the group of agents sharing a limit - with a Legion.RateLimiter.Policy. Legion calls enforce!/2 for you: configure a limiter and rules, and every turn is checked against them.

Legion.start_link(ChatAgent,
  rate_limit: [
    limiter: MyApp.RateLimiter,
    rules: [
      %Legion.RateLimiter.Rule{
        identity: %{"ip" => "203.0.113.42", "tenant" => "acme"},
        policy: %Legion.RateLimiter.Policy{
          window_ms: :timer.minutes(30),
          max_agents: 40,
          max_tokens: 200_000
        }
      },
      %Legion.RateLimiter.Rule{
        identity: %{"email" => "someone@example.com"},
        policy: %Legion.RateLimiter.Policy{window_ms: :timer.hours(24), max_agents: 5}
      }
    ]
  ]
)

A turn runs only if every rule allows it; the first rule that denies it cancels the turn. Rules are evaluated in the order given, so put the rule whose denial you want reported first.

For Postgres users there is a ready-made adapter - see Legion.RateLimiter.Postgres. It extends Legion.Store.Postgres, keeping rate-limit metadata in the store's table and counting the usage persisted there:

defmodule MyApp.RateLimiter do
  use Legion.RateLimiter.Postgres, repo: MyApp.Repo
end

Configuration

The limiter and a default policy can be set globally, leaving only the identities to the call site:

config :legion, :rate_limit,
  limiter: MyApp.RateLimiter,
  default_policy: %Legion.RateLimiter.Policy{window_ms: :timer.minutes(1), max_agents: 10}

Legion.start_link(ChatAgent,
  rate_limit: [rules: [%Legion.RateLimiter.Rule{identity: %{"ip" => "203.0.113.42"}}]]
)

A rule given without a :policy takes the default one. Rate limiting applies only when a limiter and at least one rule resolve; a limiter without rules, or rules without a limiter, disables it. Sub-agents inherit whatever their parent resolved and cannot override it - a parent started without rate limiting runs its whole subtree without it, even when the application configures rules globally. Each sub-agent is a separate agent ID in every group, so it counts towards :max_agents and its usage towards :max_tokens.

Rules must agree on their identities: two rules may share a field only with the same value, since the adapter records one group membership per agent. Two rules with the same identity and different policies are fine - for example a per-minute and a per-day window on one IP. Identity keys must be strings; an empty identity %{} matches every agent and acts as a global cap.

When Legion enforces

Enforcement happens once per turn, before the incoming message is appended and before any LLM request. A denial therefore leaves the conversation untouched and costs nothing.

Denied turns return {:cancel, {:rate_limited, violations}} - the same shape as {:cancel, :reached_max_iterations} - so a denied sub-agent reports back to its caller as a value rather than a crash. Legion also emits [:legion, :rate_limit, :exceeded] with the identity and policy of the rule that denied it; see Legion.Telemetry.

Resumed and recovered runs are not checked again. They are finishing work that was already allowed, so checking it again would discard accepted work instead of shedding new load.

Calling it yourself

resolve!/1 and enforce!/2 are public, so an application can rate-limit its own work - a webhook, a queue worker - under the same configuration as agents. Resolve the configured limiter and rules, which fills in the default policy, then enforce them under a stable id:

%{limiter: limiter, rules: rules} =
  Legion.RateLimiter.resolve!(
    rules: [%Legion.RateLimiter.Rule{identity: %{"ip" => "203.0.113.42"}}]
  )

if limiter, do: :ok = limiter.enforce!(agent_id, rules)

A nil limiter means rate limiting is off. An adapter can also be called directly with complete rules, skipping resolve!/1.

Implementing a rate limiter

Define a module that implements enforce!/2 when rate-limit state lives outside Postgres or needs application-specific coordination:

defmodule MyApp.RateLimiter do
  @behaviour Legion.RateLimiter

  @impl Legion.RateLimiter
  def enforce!(agent_id, rules) do
    # Check and record the rate-limit state for every rule, atomically.
    :ok
  end
end

An adapter receives all of an agent's rules in one call so it can allow or deny the call as a unit. See Legion.RateLimiter.Rule, Legion.RateLimiter.Policy, and Legion.RateLimiter.Postgres.