DeribitEx.RateLimitHandler (deribit_ex v0.2.0)

View Source

Adaptive rate limit handler for Deribit WebSocket API.

This module implements the WebsockexNova.Behaviors.RateLimitHandler behavior with adaptive rate limiting capabilities:

  • Dynamically adjusts rate limits based on 429 responses from Deribit
  • Implements exponential backoff when rate limits are hit
  • Prioritizes critical operations like order cancellation
  • Recovers rate limits gradually after backoff periods
  • Emits detailed telemetry for monitoring rate limiting behavior

Configuration Options

Three rate limit modes are supported:

  • :cautious - Strict limits to avoid 429s completely
  • :normal - Balanced approach
  • :aggressive - Higher throughput, might get occasional 429s

Backoff Strategy

When a 429 response is received:

  1. The handler reduces the rate limit by the recovery_factor
  2. It applies a backoff multiplier to delay subsequent requests
  3. The backoff multiplier increases with consecutive 429s
  4. The rate limit gradually recovers over time

Summary

Types

Adaptive state for the rate limiter.

Token bucket state for the rate limiter.

Configuration for the rate limiter.

Cost map specifying token costs for different operation types.

Operation types that can be rate limited.

Rate limiting modes that determine token bucket parameters.

Request handler tracking information.

Overall state for the rate limiter.

Functions

Checks if a request should be rate limited.

Handles the rate limiter tick (called periodically by WebsockexNova).

Initializes rate limit state with token bucket and adaptive parameters.

Types

adaptive_state()

@type adaptive_state() :: %{
  backoff_multiplier: float(),
  backoff_initial: float(),
  backoff_max: float(),
  backoff_reset_after: pos_integer(),
  last_429: integer() | nil,
  recovery_factor: float(),
  recovery_increase: float(),
  recovery_interval: pos_integer(),
  last_recovery: integer(),
  original_capacity: pos_integer(),
  original_refill_rate: pos_integer(),
  telemetry_prefix: [atom()]
}

Adaptive state for the rate limiter.

bucket_state()

@type bucket_state() :: %{
  tokens: non_neg_integer(),
  capacity: pos_integer(),
  refill_rate: pos_integer(),
  refill_interval: pos_integer(),
  last_refill: integer(),
  queue: :queue.queue(),
  queue_size: non_neg_integer(),
  queue_limit: pos_integer()
}

Token bucket state for the rate limiter.

config()

@type config() :: %{mode: rate_limit_mode(), cost_map: cost_map()}

Configuration for the rate limiter.

cost_map()

@type cost_map() :: %{optional(operation_type()) => non_neg_integer()}

Cost map specifying token costs for different operation types.

operation_type()

@type operation_type() ::
  :auth | :subscription | :query | :order | :high_priority | :cancel

Operation types that can be rate limited.

rate_limit_mode()

@type rate_limit_mode() :: :cautious | :normal | :aggressive

Rate limiting modes that determine token bucket parameters.

request_info()

@type request_info() :: %{
  id: String.t(),
  op_type: operation_type(),
  sent_at: integer()
}

Request handler tracking information.

state()

@type state() :: %{
  bucket: bucket_state(),
  adaptive: adaptive_state(),
  config: config(),
  response_handlers: %{optional(String.t()) => request_info()},
  last_tick: integer() | nil,
  recent_responses: [map()] | nil
}

Overall state for the rate limiter.

Functions

check_rate_limit(request, state)

@spec check_rate_limit(map() | String.t(), state()) ::
  {:allow, state()} | {:queue, state()} | {:reject, any(), state()}

Checks if a request should be rate limited.

This is the main entry point for the rate limiting handler and is called for every request sent to the WebSocket API.

Parameters

  • request - The request to be rate limited (map or JSON string)
  • state - Current rate limiter state

Returns

  • {:ok, state} - Request is allowed, with updated state
  • {:backoff, delay_ms, state} - Request should be delayed by specified ms

current_time_ms()

@spec current_time_ms() :: integer()

handle_tick(state)

@spec handle_tick(state()) :: {:ok, state()}

Handles the rate limiter tick (called periodically by WebsockexNova).

This is used for processing responses and adjusting rate limits based on 429 responses from the Deribit API.

Parameters

  • state - Current rate limiter state

Returns

  • {:ok, state} - Updated state after tick processing

rate_limit_init(opts)

@spec rate_limit_init(map()) :: {:ok, state()}

Initializes rate limit state with token bucket and adaptive parameters.

Parameters

  • opts - Configuration options for the rate limiter:
    • :mode - Rate limiting mode (:cautious, :normal, :aggressive)
    • :cost_map - Map of operation costs
    • :capacity - Override token bucket capacity
    • :refill_rate - Override token refill rate
    • :refill_interval - Override token refill interval
    • :queue_limit - Maximum queue size for delayed requests
    • :adaptive - Adaptive rate limiting options:
      • :backoff_initial - Initial backoff multiplier when 429 is hit
      • :backoff_max - Maximum backoff multiplier
      • :backoff_reset_after - Time after which backoff resets (ms)
      • :recovery_factor - Factor to reduce capacity by when 429 is hit
      • :recovery_increase - Rate at which capacity recovers
      • :recovery_interval - Interval between capacity recovery attempts
      • :telemetry_prefix - Prefix for telemetry events

Returns

  • {:ok, state} - Initialized rate limiter state