DpExchange.Core.PollingFeed (DpExchangeCore v0.1.7)

Copy Markdown View Source

A feed built from repeated fetches, for a venue package to use INSIDE its own feed.

Why this lives in the contract and not in a consumer

A venue with no streaming API still has to present a feed, or every consumer above it forks on transport — which is the drift the facade exists to undo. Robinhood has no socket and never will; its feed is this module, and nothing above the package can tell the difference.

Venues WITH sockets use it too, for the symbols their stream does not reach. That gap-filling decision belongs to the venue because only the venue can make it correctly: Webull's stream is bounded by "3 messages per second per connection", and which symbols lose that race is knowable only by watching its own sockets. Shared code had to guess, and guessed that 151 delivering-nothing symbols were a quiet market.

Shipping it in the contract rather than in a consumer means a venue package can build a feed without reaching for anything outside its own dependencies — which is the property that lets a venue be a standalone package at all.

Bulk where the venue offers it, per-symbol where it does not

A venue with an overview endpoint fetches its whole set in one request (:fetch_all). Robinhood's 87 pairs cost one call per cycle that way and 87 without, and its rate limiter is not hypothetical — dropping to per-symbol fetches would multiply this venue's request count by the size of its catalog.

Where only a per-symbol endpoint exists (:fetch), every symbol runs on its own schedule rather than the set being swept in a burst. A burst is what a rate limiter sees as an attack, and it makes the first symbol in the list permanently fresher than the last, so start times are spread across the interval to keep the request rate flat.

A fetch that fails does not stop the feed

A symbol whose fetch fails is retried on the next tick and reported as :not_covered until one succeeds. It is never dropped: this module cannot tell a delisted symbol from a network blip, and the layer that CAN — the venue answering with an explicit refusal — handles that separately.

A feed delivering NOTHING says so, loudly

Individual failures are debug-level, because a handful of them are ordinary. A feed where nothing at all is succeeding is not ordinary, and it is the most expensive failure shape in collection: it is indistinguishable from a quiet venue, which is how one venue's feed ran through its first deployment publishing nothing at all: it had been handed a credential whose key was still ciphertext. Credentials are accepted as an opaque map, so a wrong one is not an error at the boundary — it is a fetch that fails, and the per-symbol failures were invisible at debug level.

So a feed that has delivered nothing since it started escalates: it warns once it has failed a full cycle with zero successes, and keeps warning while that holds. Never a silent retry loop.

Summary

Types

Fetches one symbol's current price.

Functions

Returns a specification to start this module under a supervisor.

Which symbols this poller is currently delivering.

Whether this feed is delivering, and what went wrong if it is not.

Replace the symbol set without restarting the poller.

Types

fetch()

@type fetch() :: (String.t() -> {:ok, map()} | {:error, term()} | {:refused, term()})

Fetches one symbol's current price.

Three outcomes, and the third matters. {:ok, event} publishes. {:error, reason} is retried on the next tick — this module cannot tell a delisted symbol from a network blip, so it must not decide. {:refused, reason} is the venue stating it does not carry the symbol at all, which only the adapter can recognise, and which is reported once rather than retried forever.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

coverage(server)

@spec coverage(pid() | atom()) :: %{required(String.t()) => :internal_poll}

Which symbols this poller is currently delivering.

OBSERVED: a symbol appears only once a fetch has actually succeeded for it and is still recent. A symbol that has been asked for and never answered is absent, because reporting it as covered would be the venue asserting a delivery that never happened.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

status(server)

@spec status(pid() | atom()) :: %{
  delivering: boolean(),
  symbols: non_neg_integer(),
  covered: non_neg_integer(),
  failures_since_ok: non_neg_integer(),
  last_error: term()
}

Whether this feed is delivering, and what went wrong if it is not.

Exposed as DATA and not only as a log line, because "delivering nothing" is the condition a health check has to be able to ask about. Robinhood's feed ran a whole deployment publishing nothing — a log nobody greps in time is how that stays true for hours.

update_symbols(server, symbols)

@spec update_symbols(pid() | atom(), [String.t()]) :: :ok

Replace the symbol set without restarting the poller.