Token bucket rate limiter for HTTPower.
Implements a token bucket algorithm to enforce rate limits on HTTP requests. Each bucket refills tokens at a configured rate, and requests consume tokens.
Features
- Token bucket algorithm with automatic refill
- Per-client or per-endpoint rate limiting
- Configurable strategies: wait or error
- ETS-based storage for high performance
- Automatic cleanup of old buckets
- Support for custom bucket keys
Configuration
config :httpower, :rate_limit,
enabled: true, # Enable/disable rate limiting (default: false)
requests: 100, # Max requests per time window
per: :second, # Time window: :second, :minute, :hour
strategy: :wait, # Strategy: :wait or :error
max_wait_time: 5000, # Max wait time in ms (default: 5000)
adaptive: true # Adjust rate based on circuit breaker health (default: true)Usage
# Global rate limiting (from config)
HTTPower.get("https://api.example.com/users")
# Per-client rate limiting
client = HTTPower.new(
base_url: "https://api.example.com",
rate_limit: [requests: 50, per: :minute]
)
HTTPower.get(client, "/users")
# Custom bucket key
HTTPower.get("https://api.example.com/users",
rate_limit_key: "api.example.com"
)Token Bucket Algorithm
The token bucket algorithm works as follows:
- Each bucket has a maximum capacity (max_tokens)
- Tokens are added at a fixed rate (refill_rate)
- Each request consumes one or more tokens
- If no tokens available:
- :wait strategy - waits until tokens are available (up to max_wait_time)
- :error strategy - returns {:error, :too_many_requests}
Adaptive Rate Limiting
When adaptive: true is enabled, rate limits automatically adjust based on
circuit breaker health to prevent thundering herd during service recovery:
- Circuit closed (healthy) → 100% rate (full speed)
- Circuit half-open (recovering) → 50% rate (be gentle)
- Circuit open (down) → 10% rate (minimal health checks)
This coordination prevents overwhelming a recovering service with full traffic immediately after it comes back up.
Implementation Details
Internally this uses the GCRA (Generic Cell Rate Algorithm) formulation of a token bucket: each bucket is a single timestamp (the "theoretical arrival time") rather than a token count, which makes the hot path a lock-free compare-and-swap. The token-bucket semantics above are preserved exactly.
- Uses a public ETS table for fast in-memory storage; one integer per bucket
- Refill is implicit in the GCRA timestamp arithmetic (no periodic ticking)
- Thread-safe via lock-free compare-and-swap (
:ets.select_replace/2) in the caller process — no GenServer round-trip on the hot path; rejects write nothing - The GenServer owns only the table lifecycle and periodic cleanup of idle buckets
- Adaptive mode queries circuit breaker state (read-only, no coupling)
Summary
Functions
Checks if a request can proceed under rate limit constraints.
Returns a specification to start this module under a supervisor.
Consumes tokens from the bucket and waits if necessary.
Returns the raw GCRA state (tat_us) of a bucket, or nil if it doesn't exist.
Returns the raw GCRA state for a bucket as a map, or nil if it doesn't exist.
Feature callback for the HTTPower pipeline.
Resets a specific bucket, clearing all tokens.
Starts the rate limiter GenServer.
Synchronizes the local bucket with a server's reported rate limit state.
Types
@type bucket_key() :: String.t()
@type rate_limit_config() :: [ requests: pos_integer(), per: :second | :minute | :hour, strategy: :wait | :error, max_wait_time: pos_integer() ]
@type tat_us() :: integer()
Functions
@spec check_rate_limit(bucket_key(), rate_limit_config()) :: {:ok, float()} | {:ok, :disabled} | {:error, :too_many_requests, integer()}
Checks if a request can proceed under rate limit constraints.
Returns:
{:ok, remaining_tokens}if request is allowed{:error, :too_many_requests, wait_time_ms}if rate limit exceeded{:ok, :disabled}if rate limiting is disabled
Examples
iex> HTTPower.Middleware.RateLimiter.check_rate_limit("api.example.com")
{:ok, 100.0}
# once the bucket is exhausted:
iex> HTTPower.Middleware.RateLimiter.check_rate_limit("api.example.com", requests: 5, per: :second)
{:error, :too_many_requests, 200}Returns a specification to start this module under a supervisor.
See Supervisor.
@spec consume(bucket_key(), rate_limit_config()) :: :ok | {:error, :too_many_requests}
Consumes tokens from the bucket and waits if necessary.
This is the main function used by HTTPower.Client. It handles both :wait and :error strategies.
Returns:
:okif request can proceed{:error, :too_many_requests}if rate limit exceeded and strategy is :error{:error, :too_many_requests}if wait time exceeds max_wait_time
Examples
iex> HTTPower.Middleware.RateLimiter.consume("api.example.com")
:ok
iex> HTTPower.Middleware.RateLimiter.consume("api.example.com", strategy: :error)
{:error, :too_many_requests}@spec get_bucket_state(bucket_key()) :: tat_us() | nil
Returns the raw GCRA state (tat_us) of a bucket, or nil if it doesn't exist.
This is the theoretical arrival time in monotonic microseconds. For a
human-meaningful "tokens remaining" value, use check_rate_limit/2, which
interprets the state against a rate config.
@spec get_info(bucket_key()) :: %{tat_us: tat_us()} | nil
Returns the raw GCRA state for a bucket as a map, or nil if it doesn't exist.
Examples
iex> HTTPower.Middleware.RateLimiter.get_info("api.github.com")
%{tat_us: 1_234_567_890}Feature callback for the HTTPower pipeline.
Checks and consumes rate limit tokens for the request.
Returns:
:okif request can proceed{:error, reason}if rate limit exceeded
Examples
iex> request = %HTTPower.Request{url: "https://api.example.com", ...}
iex> HTTPower.Middleware.RateLimiter.handle_request(request, [requests: 100, per: :minute])
:ok@spec reset_bucket(bucket_key()) :: :ok
Resets a specific bucket, clearing all tokens.
Useful for testing or manual intervention.
Starts the rate limiter GenServer.
@spec update_from_headers(bucket_key(), map(), rate_limit_config()) :: :ok
Synchronizes the local bucket with a server's reported rate limit state.
Server headers report a remaining count; GCRA stores a single timestamp, so
we position the bucket's tat such that remaining tokens read back under the
given rate config (which supplies the window/rate the server count is
relative to). remaining is clamped to [0, requests].
Examples
iex> info = %{limit: 60, remaining: 55, reset_at: ~U[2025-10-01 12:00:00Z], format: :github}
iex> HTTPower.Middleware.RateLimiter.update_from_headers("api.github.com", info, requests: 60, per: :minute)
:ok