HTTPower.Middleware.Dedup (HTTPower v0.24.0)

Copy Markdown View Source

In-flight request deduplication to prevent duplicate operations.

This module prevents duplicate requests from causing duplicate side effects (e.g., double charges, duplicate orders) by tracking in-flight requests and sharing responses with identical concurrent requests.

How It Works

  1. Request Fingerprinting - Each request gets a hash based on method + URL + body, plus the auth-context headers (authorization, cookie, x-api-key) so responses are never shared across different credentials
  2. In-Flight Tracking - First request executes normally, subsequent identical requests wait
  3. Response Sharing - When the first request completes, all waiting requests receive the same response
  4. Automatic Cleanup - Completed responses are cached briefly so late-arriving duplicates still share them, then swept; orphaned in-flight rows (owner crashed without completing) are reaped as a safety net

Use Cases

  • Prevent double charges from double-clicks on payment buttons
  • Prevent duplicate orders from retry storms or race conditions
  • Ensure idempotency for critical mutations (POST/PUT/DELETE)

Configuration

# Global configuration
config :httpower, :deduplicate,
  enabled: true

# Per-request configuration
HTTPower.post(url,
  body: payment_data,
  deduplicate: true
)

# Or with options
HTTPower.post(url,
  body: payment_data,
  deduplicate: [
    enabled: true,
    wait_timeout: 60_000,   # How long waiters block for an in-flight request (default: 30s)
    key: "custom-dedup-key"  # Optional: override hash generation
  ]
)

Cache and reap windows are fixed, not per-request

The completed-response cache window (~0.5s) and the orphaned-in-flight reap ceiling (60s) are internal constants. Cleanup runs in a shared process with no per-request context, so these are not configurable per request. wait_timeout and key are the per-request knobs.

States

  • :in_flight - Request currently executing, other identical requests will wait
  • :completed - Brief window (~0.5s) after completion so late duplicates still share the response

Thread Safety

Hot path (claim, wait, complete, cancel) is lock-free on a public ETS table via :ets.insert_new/2 and :ets.select_replace/2 CAS — no GenServer round-trip. The GenServer owns the table lifecycle and runs periodic cleanup only.

Summary

Functions

Returns a specification to start this module under a supervisor.

Feature callback for the HTTPower pipeline.

Starts the request deduplicator GenServer.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

handle_request(request, config)

Feature callback for the HTTPower pipeline.

Checks for duplicate requests and either executes, waits, or returns cached response.

Returns:

  • {:ok, request} with dedup info stored in private (first occurrence)
  • {:halt, response} if cached response available (short-circuit)
  • Waits and returns {:halt, response} for duplicate in-flight requests

Examples

iex> request = %HTTPower.Request{method: :post, url: "https://api.example.com/charge", body: "..."}
iex> HTTPower.Dedup.handle_request(request, [enabled: true])
{:ok, modified_request}

start_link(opts \\ [])

Starts the request deduplicator GenServer.