LLMUtils.RateLimiter (llm_utils v0.1.3)

Copy Markdown View Source

In-memory ETS-based sliding window rate limiter.

Usage

LLMUtils.RateLimiter.check("nim", 40, 60_000)
# => :ok | :rate_limited

Override by passing a module implementing check/3 in Client options.

Summary

Functions

check(provider_id, max_requests, window_ms)

Check if a request is within rate limits.

reset(provider_id)

Reset rate limiter state for a provider.

Functions

check(provider_id, max_requests, window_ms)

@spec check(String.t(), non_neg_integer(), pos_integer()) :: :ok | :rate_limited

Check if a request is within rate limits.

  • provider_id — Provider identifier (used as ETS table name)
  • max_requests — Maximum number of requests in the window
  • window_ms — Window duration in milliseconds

Returns :ok if under limit, :rate_limited if at/over limit.

reset(provider_id)

@spec reset(String.t()) :: :ok

Reset rate limiter state for a provider.