DpExchange.Core.HttpClient (DpExchangeCore v0.1.7)

Copy Markdown View Source

The shared HTTP request pipeline: rate limiting, retry, logging, error shaping and the generic authentication schemes.

What it deliberately does not know

No venue appears in this module. It once carried a Coinbase JWT builder and a case provider do table mapping "coinbase" and "gemini" to their own rate-limit header parsers. Both moved into their venue packages, because a venue fact living in shared code is a second place that can be wrong about that venue — and it was: every provider except Coinbase once fell through to Coinbase's socket codec, so one venue's socket spoke another's protocol at its own endpoint and delivered nothing for as long as that stood.

What is left is generic: :hmac_sha256, :basic and :bearer auth, the request pipeline, query-string building, response parsing, retry, and a rate-limit header parser that reads only the conventional x-ratelimit-* shape. A venue whose scheme or headers differ supplies its own — see build_auth_headers/5 and parse_rate_limit_headers/1.

The limiter is resolved at call time

Never at compile time. Application.compile_env/3 would freeze whichever limiter the consumer configured when this dependency was compiled, so a consumer changing it later would either have to recompile or get a boot-time mismatch. Resolution goes through DpExchange.Core.Config, so a consumer's async: true test can swap the limiter for its own process tree without configuring it for every test beside it.

Neither acquire nor check fills the bucket

They answer "is there capacity" and, for acquire, wait until there is. Something has to report what actually left, or the bucket stays empty and every check passes. This module records on behalf of every request it makes; a venue package issuing its own HTTP calls must record for itself. One that did not acquired before every request and recorded none, so its ceiling bound nothing: 395 calls per 60s against a documented 300, while the budget panel read 83/240.

Summary

Types

How to authenticate: one of the generic schemes, or a venue's own builder.

A response as it comes back from the pipeline.

Why a request did not produce a response.

Functions

Build authentication headers for API requests.

Convenience function for GET requests with query parameters.

Parses rate-limit information from response headers, for the conventional x-ratelimit-limit / x-ratelimit-remaining / x-ratelimit-reset shape only.

Make an HTTP request with provider-specific and account-aware rate limiting.

Types

account_id()

@type account_id() :: String.t()

auth_scheme()

@type auth_scheme() ::
  :hmac_sha256
  | :basic
  | :bearer
  | (http_method(), String.t(), body(), map() -> headers())

How to authenticate: one of the generic schemes, or a venue's own builder.

The function form is the hook that replaced a per-venue branch. The spec named only atom() for a while after the hook was added, so a venue passing a builder — the whole point of the hook — was told by dialyzer that the call "breaks the contract". Adding a capability without widening its spec makes the tool argue against the feature.

body()

@type body() :: String.t() | nil

headers()

@type headers() :: [{String.t(), String.t()}]

http_method()

@type http_method() :: :get | :post | :put | :delete

http_response()

@type http_response() :: %{
  status: integer(),
  headers: headers() | map(),
  body: term()
}

A response as it comes back from the pipeline.

body is whatever the transport decoded, not a string. Req decodes JSON into a map or a list before this module sees it, and only leaves a binary when it could not.

It was declared String.t(), and that was not a harmless inaccuracy: dialyzer then concluded that any consumer matching a decoded body was matching something impossible, so every function reachable only through that branch was reported as unreachable dead code. A venue package's mix dialyzer failed with Function decimal/1 will never be called about a function called on every price it parses.

The application this pipeline came from hit the same class of problem from the other end and said so in a comment: a missing clause "poisoned dialyzer's success typing for every HttpClient.request/5 caller — the whole chain got narrowed to {:error, _} only." A wrong type is worse than a missing one; it makes the tool confidently wrong, and the reader believes it.

options()

@type options() :: keyword()

provider()

@type provider() :: String.t()

rate_limited_request_options()

@type rate_limited_request_options() ::
  keyword()
  | %{
      provider: provider(),
      account_id: account_id(),
      user_id: user_id(),
      operation: String.t()
    }

request_error()

@type request_error() :: String.t() | {:exchange_error, provider(), term()}

Why a request did not produce a response.

Three shapes, and the spec used to name only the first — which made dialyzer tell every consumer that its handling of the other two was unreachable dead code.

  • String.t() — a plain message, when no :provider was given.
  • {:exchange_error, provider, reason} — the same message tagged with the venue, which is what a caller gets whenever it passes :provider, i.e. almost always.
  • The rate-limit case is a three-element tuple, {:error, :rate_limited, retry_after: seconds}, not a two-element one. Unusual, and worth stating rather than leaving a caller to discover it from a CaseClauseError.

user_id()

@type user_id() :: String.t()

Functions

build_auth_headers(method, path, body, credentials, auth_type)

@spec build_auth_headers(http_method(), String.t(), body(), map(), auth_scheme()) ::
  headers()

Build authentication headers for API requests.

Parameters

  • method: HTTP method
  • path: API endpoint path
  • body: Request body
  • credentials: API credentials
  • auth_type: Authentication type (:hmac_sha256, :basic, :bearer)

Returns

  • List of authentication headers

get(url, params \\ [], opts \\ [])

@spec get(String.t(), keyword() | map(), rate_limited_request_options()) ::
  {:ok, any()} | {:error, String.t()}

Convenience function for GET requests with query parameters.

Parameters

  • url: Base URL for the request
  • params: Query parameters as keyword list or map
  • opts: Additional options

Returns

  • {:ok, parsed_json} on success
  • {:error, reason} on failure

parse_rate_limit_headers(headers)

@spec parse_rate_limit_headers(headers() | map()) :: map() | nil

Parses rate-limit information from response headers, for the conventional x-ratelimit-limit / x-ratelimit-remaining / x-ratelimit-reset shape only.

Returns nil when the headers do not carry it. nil means "this response did not say", never "there is no limit" — a caller must not read an absent header as headroom.

This used to take a provider name and dispatch on it, with branches for two venues' bespoke headers. Those branches are venue knowledge and moved into the venue packages (D-C); a venue whose headers differ parses them itself, and does not need this function to have heard of it.

request(method, url, headers \\ [], body \\ nil, opts \\ [])

@spec request(
  http_method(),
  String.t(),
  headers(),
  body(),
  rate_limited_request_options()
) ::
  {:ok, http_response()}
  | {:error, request_error()}
  | {:error, :rate_limited, [{:retry_after, non_neg_integer()}]}

Make an HTTP request with provider-specific and account-aware rate limiting.

Parameters

  • method: HTTP method (:get, :post, :put, :delete)
  • url: Full URL for the request
  • headers: List of HTTP headers
  • body: Request body (for POST/PUT requests)
  • opts: Additional options including rate limiting context

Options

  • :timeout - Request timeout in milliseconds (default: 30_000)
  • :retry_attempts - Number of retry attempts (default: 3)
  • :retry_delay - Base delay between retries in milliseconds (default: 1000)
  • :log_requests - Whether to log requests (default: true)
  • :provider - Provider name for rate limiting (required for rate limiting)
  • :account_id - Account ID for account-aware rate limiting (optional)
  • :user_id - User ID for additional isolation (optional)
  • :operation - Operation type for fine-grained rate limiting (default: "default")

Returns

  • {:ok, http_response()} on success
  • {:error, reason} on failure