LemonAi.RateLimiter (lemon_ai v0.1.0)

View Source

Token bucket rate limiter GenServer for managing per-provider request rates.

Overview

This module implements a token bucket algorithm where tokens are added at a configurable rate up to a maximum capacity. Each request consumes one token, and requests are rejected when no tokens are available.

Usage

# Start the rate limiter (typically via supervision tree)
{:ok, pid} = LemonAi.RateLimiter.start_link(
  provider: :anthropic,
  tokens_per_second: 10,
  max_tokens: 20
)

# Check if a request is allowed
case LemonAi.RateLimiter.acquire(:anthropic) do
  :ok -> # proceed with request
  {:error, :rate_limited} -> # back off
end

# Return a permit after request completes (optional, for concurrency tracking)
LemonAi.RateLimiter.release(:anthropic)

Configuration

  • tokens_per_second - Rate at which tokens are replenished (default: 10)
  • max_tokens - Maximum bucket capacity (default: 20)
  • provider - Provider identifier (required)

Summary

Types

provider()
state()

Functions

acquire(provider)

Attempt to acquire a permit for the given provider.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

ensure_started(provider, opts \\ [])

Ensure a rate limiter exists for the provider.

get_state(provider)

Get current state for debugging/monitoring.

release(provider)

Release a permit back to the limiter.

start_link(opts)

Start a rate limiter for a provider.

Types

provider()

@type provider() :: atom()

state()

@type state() :: %{
  provider: provider(),
  tokens: float(),
  max_tokens: pos_integer(),
  tokens_per_second: pos_integer(),
  last_refill: integer()
}

Functions

acquire(provider)

@spec acquire(provider()) :: :ok | {:error, :rate_limited}

Attempt to acquire a permit for the given provider.

Returns :ok if a token was available, or {:error, :rate_limited} if the bucket is empty.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

ensure_started(provider, opts \\ [])

@spec ensure_started(
  provider(),
  keyword()
) :: {:ok, pid()} | {:error, term()}

Ensure a rate limiter exists for the provider.

Starts a limiter under LemonAi.ProviderSupervisor when available.

get_state(provider)

@spec get_state(provider()) :: {:ok, map()} | {:error, :not_found}

Get current state for debugging/monitoring.

release(provider)

@spec release(provider()) :: :ok

Release a permit back to the limiter.

This is a no-op for token bucket rate limiting (tokens auto-refill), but can be used for tracking active requests.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Start a rate limiter for a provider.

Options

  • :provider - Provider identifier (required)
  • :tokens_per_second - Token refill rate (default: 10)
  • :max_tokens - Maximum bucket capacity (default: 20)