DpExchange.Robinhood.Feed (DpExchangeRobinhood v0.3.2)

Copy Markdown View Source

This venue's feed — a REST poll, and nothing outside this module needs to know that.

Why a venue with no socket still has a feed

Robinhood Crypto exposes no streaming API. Under the shape this replaces, that fact travelled upward: the collection layer kept a poll set and decided which venues were exempt from it, and an operations page described Robinhood's pairs in terms of a socket it does not have and has never claimed — sending a reader hunting a streaming fault that cannot exist.

Behind a feed, the poll is an implementation detail. This module delivers the same Core.Types.TopOfBook to the same subscriber as a WebSocket venue, so no consumer branches on transport, and coverage/1 can report what the venue actually reports about itself: these symbols are arriving.

Not Core.Types.Quote. This venue has no last-trade endpoint at all — best_bid_ask carries only bid and ask — and DpCryptoManagement's issue #21 is what happens when this polled Core.Types.Quote.price from the ask to paper over that: a fabricated trade price masquerading as a real one. See DpExchange.Robinhood's moduledoc on get_price/2. Bid and ask are both genuine, so that is what this polls and delivers.

One request per cycle, not one per symbol

This runs Core.PollingFeed in its bulk :fetch_all mode: every tick sends ONE signed request carrying every symbol in scope, via Rest.get_top_of_book_bulk/3, rather than one signed request per symbol. At the ~86-pair catalogue this package inherited, that is the difference between roughly 86 requests a cycle and 1.

Correction, 2026-09-06: an earlier version of this note said the venue "publishes no bulk-stats endpoint" and left it there. That is not quite what the vendor's own OpenAPI document says. best_bid_ask genuinely carries no 24-hour statistics — that part holds — but its symbol query parameter is documented as repeatable: ?symbol=BTC-USD&symbol= ETH-USD returns a results array covering every symbol asked for in ONE signed request. This module ran per-symbol for a time after that correction, recorded as docs/design/ideas/bulk-best-bid-ask-fetch.md, because two things were true: Core.PollingFeed's own moduledoc names Robinhood as the intended user of its :fetch_all mode for exactly this shape, but the vendor's document never says what a batched call does when one symbol in it is unlisted or malformed — drop that row and 200 the rest, or 400 the whole request — and PollingFeed's fetch_all path had no on_refusal-equivalent, so a {:refused, _} from a bulk fetcher would crash this feed's process instead of recording one refused symbol. dp_exchange_core gained that handling (fetch_all_and_publish/1's {:refused, refusals} clause), which closed the second condition, but the first — what the venue actually does on a partial-bad batch — is still not stated anywhere this package can read.

So this does not guess. fetch_all/3 below sends the bulk request and reads whatever comes back:

  • A results array with every row filled: published, same as before, at 1/86th the request cost.
  • A results array SHORTER than what was asked for: also published, as-is. A missing row is silence — the venue not answering for that symbol on THIS request — not a venue statement that the symbol does not exist, and PollingFeed already treats a symbol absent from a bulk response as uncovered-and-retried, never as refused (see its own publish_and_record/2). Turning silence into a refusal is the exact defect DpCryptoManagement issue #25 fixed on the single-symbol path (Rest.first_result/1's own moduledoc), and this path must not reintroduce it on the bulk one.
  • The WHOLE request refused (400/401/403/404) — the shape a single bad symbol could plausibly produce, and the one this module used to have no safe answer for. fetch_all/4 falls back to one signed request per symbol, for this cycle only, via the same Rest.get_top_of_book/3 the old per-symbol design used. That fallback reports every refusal it finds by CALLING the same on_refusal function PollingFeed itself would have called, directly, rather than by returning {:refused, refusals} — see fetch_all/4 and fallback_per_symbol/5 for why the return channel cannot carry both the refusal and the other symbols' events in one outcome. So the bad symbol is reported (once per cycle it stays in scope, same as the venue's own per-symbol refusal already behaves when a consumer never drops it) AND every other symbol that answers fine publishes in the SAME cycle — no data withheld, no cycle lost. The unavoidable cost is real but different: this cycle spends one request per symbol instead of one for the whole batch, for as long as the refused symbol stays in scope. Once a consumer reacts to the refusal and drops the symbol, the very next tick is back to one request.
  • The whole request merely erroring (a 5xx, a network failure) is left alone: retried next tick the ordinary way, at the ordinary one-request cost. Falling back per symbol here would not identify anything — an outage affects every symbol on either path alike — and would spend 86 requests to learn nothing a plain retry does not already cover.

No single bad symbol can make this feed deliver nothing indefinitely: the worst case is one degraded cycle before the offending symbol is out of scope, never a permanent whole-batch failure.

acquire, not check

A moduledoc worth carrying from the adapter this replaces. When rate limiting was first switched on for this venue — it had never been enabled at all — Robinhood went from 87 of 87 symbols delivering to 8 of 87 in a single cycle. Not the venue throttling: our own limiter refusing calls the venue was perfectly happy to serve, because check/3 answers "is there capacity right now" and a poll that finds none simply skips the symbol.

acquire/3 waits for capacity instead. A slower cycle rather than a missing price.

A silent outage says so, not only to the log

This is the venue Core.PollingFeed's "delivered NOTHING" warning was written about: DpCryptoManagement's issue #21 is a wrong credential (ciphertext where a key belonged) producing a fetch failure on every symbol, every cycle, for a whole deployment, with the only trace a Logger.warning a human had to go grepping for. dp_exchange_core 0.1.50 gives PollingFeed.start_link/1 an :on_notice option for exactly this, and it is wired here the same way on_refusal already is: forwarded into this process, then fanned out to every registered notice subscriber (see below), so a coverage outage reaches more than a log line nothing downstream reacts to. It fires once on the transition into delivering-nothing (severity: :warning) and once on the transition back out (severity: :info) — never per tick and never per sweep while the outage continues, so an 86-symbol feed retrying every symbol every cycle does not turn one outage into a notice storm.

Documenting that design was not the same as wiring it. :rate_limit_blocking — the option Core.HttpClient.check_rate_limits/1 actually reads to choose acquire/3 over check/3 — was missing from this module's own forwarded-options allowlist, so no caller could ever turn it on: every request fell through to check/3 regardless, and the failure this section describes reproduced exactly, live (DpCryptoManagement's issue #16). Forwarded now, and defaulted to true here specifically — not in Rest's own allowlist, which a direct one-off get_top_of_book/2 call also goes through and where fail-fast may be exactly what a caller wants. A poll is not a one-off call: this module's whole reason to exist is the venue's rate limit, so acquire is the only correct default for it.

A monitoring pid does not have to be the data subscriber

start_link/1's :subscriber is the single fixed pid that receives quotes, refusals and (until this section) notices — set once, at supervision-tree boot, because this venue's subscribe/2 takes no to: of its own; there is no per-call registry for market data here, family-wide or otherwise. Notices are different on purpose: subscribe_notices/2 — and DpExchange.Robinhood.subscribe_notices/1 above it — adds a genuinely independent pid to notice_subscribers, so a monitoring process that never wants a Core.Types.TopOfBook can still learn this feed went dark, without displacing whoever is already registered to receive the quotes.

This module previously claimed that registry did not exist, on the reasoning that Core.PollingFeed itself has no notion of more than one recipient — sink, on_refusal and on_notice are each exactly one injected function, by design (see Core.PollingFeed's own moduledoc). That is still true and is not being fought here: the fan-out lives in THIS module, one layer up, exactly the way dp_exchange_schwab's own Feed already fans its Streamer and fallback-poll notices out to more than one registrant. PollingFeed still owns fetching, scheduling and its own "delivering nothing" latch; this module owns nothing more than "who gets told," which is the part a single injected function structurally cannot express. Turning this module into a GenServer in its own right — where it used to simply be the PollingFeed process, registered under this module's name — is the smallest change that gives it somewhere to keep that set.

A crashed poller used to be Feed's crash too — and now it is caught

PollingFeed.start_link/1 runs inside init/1, which links the poller to this process the way start_link always does. Before this fix, nothing here trapped exits, so an abnormal poller exit — this venue has no socket to crash instead, so the poller is the only linked child there is — sent an untrappable EXIT signal along that link and crashed Feed too, restarted by DpExchange.Robinhood.Supervisor from the static opts it was given at tree-start: any symbols added since boot via update_symbols/2, and every subscribe_notices/1 registration, silently reverted. Feed traps exits now, restarts the poller with the symbol set it actually had — tracked in state.symbols, updated on every update_symbols/2 call, precisely so a crash-restart has something truer to rebuild from than the opts this process started with — and reports a :link_down Core.Notice rather than leaving the crash silent.

Summary

Functions

Returns a specification to start this module under a supervisor.

Which symbols are actually arriving. Observed, never intended.

coverage/1, split by kind — see DpExchange.Robinhood.coverage_by_kind/1 for why the family wants this at all when Robinhood has nothing to split.

Registers opts[:to] (default: the caller) to receive this feed's own Core.Notice traffic — currently the coverage-outage pair described in this module's moduledoc.

Replaces the polled set.

Functions

child_spec(init_arg)

@spec child_spec(keyword()) :: Supervisor.child_spec()

Returns a specification to start this module under a supervisor.

See Supervisor.

coverage(feed)

@spec coverage(GenServer.server()) :: %{required(String.t()) => :internal_poll}

Which symbols are actually arriving. Observed, never intended.

coverage_by_kind(feed)

@spec coverage_by_kind(GenServer.server()) :: %{
  top_of_book: %{required(String.t()) => :internal_poll}
}

coverage/1, split by kind — see DpExchange.Robinhood.coverage_by_kind/1 for why the family wants this at all when Robinhood has nothing to split.

Traceable to the actual struct, not assumed from the declared kind list: this feed's fetch calls only Rest.get_top_of_book/3, wired in init/1 below, and that function returns exclusively DpExchange.Core.Types.TopOfBook.t() — never DpExchange.Core.Types.Quote.t() (see this module's own moduledoc on why not). Every symbol coverage/1 reports therefore arrived through that one fetcher, so wrapping its map under :top_of_book reports what was actually produced, not a guess.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

subscribe_notices(feed, opts \\ [])

@spec subscribe_notices(
  GenServer.server(),
  keyword()
) :: :ok

Registers opts[:to] (default: the caller) to receive this feed's own Core.Notice traffic — currently the coverage-outage pair described in this module's moduledoc.

Additive, never a replacement: the fixed :subscriber given to start_link/1 keeps receiving notices too, exactly as it did before this registry existed. A dead pid or an unregistered name is skipped at delivery time rather than raised on — see fan_out/2.

update_symbols(feed, symbols)

@spec update_symbols(GenServer.server(), [String.t()]) :: :ok

Replaces the polled set.