The shape
defmodule DpExchange.YourVenue do
@behaviour DpExchange.Core.Venue
endThat module is the entire public API of your package. Everything else — transport, signing, session handling, supervision — is internal, and the conformance suite asserts it.
Declare the behaviour. The compiler's missing-callback check is the cheapest assertion in the whole contract and it runs before any test does.
Do not add functions to the facade
A venue does not add functions; it declares which ones it answers. A public function only your venue has is a function a caller must know which venue it is holding to call — which is the coupling the family exists to remove.
This is not hypothetical. One venue shipped get_staking_balances/2 as a public
venue-specific function. It does not cross the facade, and the consumer loses that call at
migration.
If a capability is genuinely missing from the facade, that is a Core change with a deliberate release behind it, not a local addition.
Your job is to get the data and keep the connections up — not to process it
Market data passes through. What a package may hold is state describing what it is doing, and whether it is fulfilling what the host asked for.
That is the whole boundary, and it is worth stating as its own rule rather than leaving it
to be inferred from the shapes the contract happens to offer, because a contract with only a
snapshot type and no incremental one reads — wrongly — as an invitation to accumulate. It is
not one. Decoding a venue's frames into Core.Types.* and delivering them to your
subscribers is the job. Reconstructing the venue's own state from a stream of updates is a
second job nobody asked your package to do, and the host is doing it already — this data is
already flowing into the host's own store.
Hold — bookkeeping about your own job:
- Which pairs are subscribed to which shard or connection. This is the one piece of market topology you are allowed to keep, because it is not market data — it is a record of what the host asked for and where you put it.
- The delivery and coverage tracking backing
coverage/1andcoverage_by_kind/1— whether a symbol is actually arriving, by which route, and by which kind. This is a fact about whether your package is doing its job, not a fact about the market. - A venue catalogue or alias map needed to attribute an inbound frame to the symbol the host subscribed to under. Necessary to route the frame at all; still not the market's own state.
Do not hold — reconstructions of what the venue itself is doing:
- An order book.
Types.OrderBookDeltaexists so a book stream passes through as the venue's own deltas instead of being folded into one. - A running last price, an accumulating candle, or any other rebuild of venue state from a stream of updates. If a caller needs a maintained view, it builds one on its own side of the facade, from the values you already hand it.
Why the line is drawn exactly here. The host is already streaming this same data into a
time-series store of its own, so a package-side copy is duplicated state sitting in the one
place that can least afford to hold it — and holding it costs the job the package actually
has. dp_exchange_coinbase's Socket was the one package in this family that got this
wrong: it held a full order book per symbol and rebuilt the whole thing on every delta,
which cost 65–110 ms per frame. That work ran inside the same process responsible for
WebSockex.send_frame/2, so a socket that was never idle rebuilding a book it was never
asked to keep could not service its own sends — which is the :send_timeout behind issue
#22. Maintaining state we were not supposed to hold is what broke the connections we were
supposed to keep. The other four venue packages in this family already decode a frame and
pass it on without holding a copy; this section states the convention they already follow,
for the one package that did not.
capabilities/0 is a claim about a real venue
def capabilities do
DpExchange.Core.Capabilities.new(
endpoints: %{
{:get_price, 2} => :experimental,
{:get_transfers, 2} => :unsupported
},
supported_quotes: ~w(USD USDC),
historical_timeframes: ~w(1m 1h 1d),
credential_benefit: :higher_ceiling,
public_ceiling: %{limit: 10, per_ms: 1_000},
authenticated_ceiling: %{limit: 100, per_ms: 1_000},
measured_at: ~D[2026-08-27],
measured_against: "GET /api/v3/exchangeInfo"
)
endBuild it with new/1. Assembling %Capabilities{} directly skips every validation,
and the validations are the point.
Declare what you measured, not what you assume
If a value was measured, say when and against what. If it was read from documentation and never probed, say that instead. An unlabelled number is worse than a missing one.
This is not fussiness. A state table drafted by people who knew the system was wrong in 7 of 21 rows; the measured version replaced it.
The declaration and the behaviour may not disagree
:provenor:experimental→ the function works. It may not answer{:error, :not_supported}.:unsupported→ the function exists and returns{:error, :not_supported}. Not a raise, not undefined, not degraded data.
Over-declaring fails in your caller's hands at runtime. Under-declaring hides working functionality. The suite checks both directions because checking one leaves the other open.
Never declare transport
There is no has_websocket and there must never be one. Both endpoints exist on every
venue. What a caller legitimately needs is which kinds of data stream — streamable: [:quotes, :order_book] — not which channels carry them. "level2" is your venue's word;
:order_book is everyone's.
The domain vocabularies are closed lists, and new/1 checks them
supported_order_types and supported_time_in_force are validated against a fixed list
at new/1 time — a value outside it raises rather than being carried through silently.
Read the current lists from DpExchange.Core.Capabilities's source when in doubt; they
grow only when a venue proves it needs a word this contract does not yet have.
Current supported_time_in_force: :gtc, :ioc, :fok, :gtd, :day, :gfw, :gfm.
The last two — "good for week" and "good for month" — are real Robinhood values, added
because the vendor's own OpenAPI schema names them in both the order request and response
schemas and this contract had no slot for them before. Purely additive: a venue declaring
a subset of this list is unaffected by the addition.
Current supported_order_types: :market, :limit, :stop, :stop_limit,
:post_only, :ioc, :fok, :trailing_stop, :trailing_stop_limit,
:market_on_close, :limit_on_close. The last four exist because Schwab accepts them and
Core had no word for them; declaring one says the venue accepts the type, not that Core
can express every parameter it takes — place_order/3's request map is for that.
A list-valued capability is a union across asset classes — and the call must fail closed
If your venue serves more than one asset class, streamable, authenticated_streamable and
historical_timeframes are flat lists with no class dimension, and the rule is:
A value belongs in the list if the venue serves it on any path this package reaches.
That is only honest because of the second half:
The per-call path must fail closed for a combination it does not serve. Declare 1w
because your equity bars serve it, then quietly return a 1d bar when a caller asks for
1w on crypto, and you have built the exact substitution this family exists to stop.
dp_exchange_webull is the worked example — it declares 1w/1M for the equity, option and
futures bars, and get_historical_prices/5 answers
{:error, {:unsupported_timeframe, _}} for a crypto category.
Both multi-asset venues chose this reading independently before it was written down, which is why the ambiguity went unnoticed. Do not choose the other one.
The limitation it leaves: a consumer cannot ask "which widths for crypto" or "is the
book streamable for options" — they get the venue-wide answer and learn the per-class truth
from a refusal. Two packages hit this today. Closing it means an asset-class dimension on a
published type, for a gap no consumer has reported hitting; see
DpExchange.Core.Capabilities' moduledoc for the two instances.
Prefer Types.*.new/1 over a struct literal in your decoder
Every Core.Types.* module exposes a validating new/1, built on
DpExchange.Core.Types.Validate: struct!/2, plus a check that every field the type's own
@enforce_keys names is present and non-nil, raising ArgumentError naming the
offending field when it is not.
@enforce_keys alone guards presence, not nil — %Candle{open: nil, high: ..., low: ..., close: ..., ...} builds without complaint even though Candle's typespec calls open a
Decimal.t(), never a Decimal.t() | nil. That gap is not academic: a nil in a field the
typespec forbids is exactly what a decode bug on a venue key that got renamed produces, and
without new/1 the failure surfaces several calls downstream — inside Decimal or
similar — with nothing pointing at which field was actually the problem.
%Candle{...} and every other struct literal still work; nothing here removes defstruct
or @enforce_keys, and internal code or a test building a known-good value by hand is
unaffected. new/1 is the path your own decoder should prefer, because it turns a decode
bug into an ArgumentError at the boundary instead of a crash three calls downstream with
no indication which venue field caused it.
Types.Order is the one type where this needs a caveat: it enforces the presence of seven
keys, but its own moduledoc documents that all but :provider legitimately admit nil —
"the venue's word, or nothing," since a venue can acknowledge a cancel with an id and
nothing else. So Order.new/1 narrows its check to :provider alone. Check a type's own
moduledoc rather than assuming every enforced key must come out non-nil.
Fail closed; never substitute
The recurring failure in this family is a nearby substitute where there should be an error. A missing granularity becoming the closest one. A missing endpoint becoming synthetic data. Every value stays plausible and only the meaning is wrong, which is why it does not surface as a failure.
If asked for a timeframe you do not serve, return an error. Do not serve the nearest width.
AdapterContract's assertion 21 checks this against your Fake: it picks a width from
Timeframe.nameable/0 your capabilities().historical_timeframes does not name and asks
get_historical_prices/4 for it, which must not answer {:ok, _}.
Timestamps are the venue's own
Use what the venue gave you, unchanged. Where it gave nothing, nil is the honest answer —
a substituted local clock is a plausible value with the wrong meaning.
Balance is the one exception, and it is stated: its timestamp is when you asked,
because a balance has no venue event time and its freshness is the only thing a caller can
reason about.
Carry the incident, not just the code
Where a moduledoc explains why a guard exists, that explanation is the most valuable thing in the file. Carry it when the code moves or is copied. A guard without its reason reads as defensive padding, and the next person tidying up deletes it.
The surface is 88 callbacks, and almost all of them are optional
Venue.required_callbacks/0 is the list the compiler enforces; everything else is declared
:unsupported and answers {:error, :not_supported}. A new package does not implement 88
functions. It implements what its venue serves and declares the rest — which is exactly the
work, because the declaring is where the thinking is.
One of the optional ones is optional for a different reason than the rest: coverage_by_kind/1
is not ceremony to skip, it is a callback Core ships ahead of any venue adopting it, so
that publishing it never breaks a venue package mid-release. Adopt it when you can — see
usage-rules/feeds.md for the incident it exists to make visible — but a package that has not
yet is not a package doing anything wrong.
Venue.peripheral_endpoints/0 names the ones a consumer can live without, with the reason
for each. It is what tells a package author which absences are survivable and which will
cost a consumer the migration.
Options: opts is the venue's own vocabulary, and that is deliberate
The facade takes keyword() on nearly every callback, and packages read venue-specific keys
out of it — category: on Webull, portfolio: on Coinbase, account_number: on Robinhood.
That looks like the coupling this family exists to remove, and it is not, because of one
rule:
A caller that passes no options must get a correct answer. Options select among things the venue offers; they never carry something the call cannot work without. Where a venue genuinely requires a parameter this contract has no word for — Robinhood v2's account number — the package refuses locally, by name, rather than sending a request the venue will reject with something less specific.
Two things follow that are worth stating because they are easy to get backwards:
- Never route on an option the caller did not pass. Webull's five categories are five separate endpoints with five different parameter sets; guessing which one a caller meant produces a plausible answer from the wrong market.
- An option this package does not recognise is ignored, not an error. A consumer moving between venues carries options that only one of them reads, and refusing them would make the uniform facade unusable for the thing it is for.
A forwarded opts turns "never configured" into key: nil, not into absence
Every venue package in this family forwards its own opts unchanged, by convention,
through several layers — a Feed passes its opts straight to PollingFeed.start_link/1,
which never itself set interval_ms. When nothing upstream ever configured a key, it does
not vanish from the list; it arrives as key: nil, explicit and present, because something
upstream read it with a bare Keyword.get/2 and passed the nil straight through.
Keyword.get(opts, key, default) only substitutes default for an ABSENT key, never for
one that is present and nil. Against interval_ms: nil it returns nil, not a sane
default — and a nil reaching Process.send_after/3, or arithmetic further downstream,
crashes the calling process, which this library does not supervise.
DpExchange.Core.Config.opt/3 is Keyword.get/3 with exactly that one difference: a
present-and-nil value is treated the same as an absent one. Reach for it, not
Keyword.get/3 or ||, at every default-bearing option your decoder or Feed reads out of
forwarded opts — deliberately not ||, because || is falsy on false too and would
silently turn an explicit log_requests: false back into its default.
A call with nothing waiting on it must set rate_limit_blocking: true
Core.HttpClient reads :rate_limit_blocking to decide between acquire/3 — wait for
capacity — and check/3 — fail immediately if there is none. It defaults to false,
because a caller a human is waiting on should get a fast, honest refusal rather than an
unexplained pause.
That default is wrong for every call made from a timer, a Process.send_after/3, or a
background task, and getting it wrong there is not a slow call — it is a permanent silent
degradation, because the work that was supposed to happen simply does not, and nothing is
blocked to notice.
This has now been the same defect three separate times, in three packages:
| what failed | cost | |
|---|---|---|
| issue #16 | Robinhood's Feed never forwarded the option | 87 of 87 symbols dropping to 8 in one cycle |
| issue #23 | Webull's Feed, Subscription and Rest each stripped it from their allowlists | 58 throttle failures in 13 minutes; 0 of 342 pairs streaming |
| issue #26 | Coinbase's alias-map fetch never set it | alias resolution permanently off; 406 pairs requested, 5 delivered under the requested names |
Every one is a background call, with nothing waiting on the result, failing rather than waiting a second — while Core's own throttle message names the fix in the text it returns. Reading that message requires already having shipped the bug, which is why it is written down here instead.
So, when you add a call:
- Ask who is waiting on the result. Nobody? Set
rate_limit_blocking: true. A one-second wait on a 60-second timer is free; a failure is total. - Set it at the layer that knows. A
Feedknows its resubscribe is unattended and should default it totrue; aRestmodule does not know whether its caller is a background replay or a user-facing one-off, so it should forward the option and never default it.dp_exchange_robinhood's anddp_exchange_webull's feeds are the worked examples. - Check every allowlist between you and
HttpClient. Webull's took three:Feedbuilt the options,Subscriptionfiltered them,Restfiltered them again. A fix that stops at the first layer passes every test that asserts "the keyword list contains the option" and changes nothing on the wire.
An internal function nothing in lib/ calls is a defect, not a stub
"Mechanism built, documented, and never wired." Six instances in one week across this family, every one shipped green because a test called the function directly:
| what was built | what never called it | |
|---|---|---|
| issues #16, #23, #26 | rate_limit_blocking plumbed through Core.HttpClient | the caller — three separate packages, #23 through three separate option allowlists |
| issue #22 | FrameSender's retry path — the moduledoc says a slow socket "becomes a failed batch, which a caller can report and retry" | the caller reported and never retried |
| — | dp_exchange_schwab's subscribe_notices/1 facade, backed by Feed's notice registry | the facade, which discarded opts[:to] and answered :ok unconditionally |
| — | dp_exchange_schwab's Auth.refresh/2 | everything — zero call sites in lib/, while Socket held a token that could only expire, and websockex reconnects with no delay |
Every one of those functions was reachable from a test. Coverage stayed green and the suite stayed silent, because a test is not a caller.
Core.AdapterContract's assertion 16 — "internal wiring" — checks the thing a human
reviewer checks by eye and a test suite cannot: does anything in this package's own
lib/ actually reach this function. It reads DpExchange.Core.UnwiredCheck's call
graph from :xref, the same OTP tool assertion 7 already reads for the purity check, so
a captured &Mod.fun/1 or a literal apply(Mod, :fun, args) counts as real usage the
way a grep never would. What it excludes — the facade, the fake, every behaviour's own
callbacks, child_spec/1 and 2, start_link/1, and every compiler-injected export —
is read from the same venue:/fake: bindings and behaviour declarations your contract
test already supplies, never a hand-maintained list. See Core.UnwiredCheck's
moduledoc for the full account of what it does and does not catch, including the one
real gap: a function reachable only through your own Fake and dead on every real path
is not flagged, because Fake is part of lib/ too.
When assertion 16 fails, the finding names a real function at a real line. Read it as one of two things, never a third: either the function is genuinely dead and should go, or it is exactly the shape above — reachable, tested, and never actually wired to the thing that was supposed to call it. Wire it, or delete it. Do not add it to an exclusion list; there is no per-function list to add it to.
Assertion 17 — your Fake must not be more capable than your real venue
Found independently in two venue packages the same week this assertion was added: six
credentialed functions on a venue where every request is signed and there is no
anonymous endpoint answered {:ok, _} for %{} or nil credentials, because nothing
in the fake checked the argument at all. Tier 1 (in-process fakes) is the only tier that
runs on every CI run and the only one most consumers ever exercise — a fake that
succeeds where the real venue would refuse silently certifies consumer code that forgot
to supply credentials.
If your capabilities/0 declares credential_benefit: :required, assertion 17 calls
every active endpoint on your fake: with every credential-bearing argument
stripped and refuses {:ok, _} back. It is Fake-only — it never dials the real venue —
so it carries none of the risk a live-network assertion would.
"Every active endpoint", not "every endpoint that takes a credentials() argument".
:required means every active endpoint on your venue needs a credential — that is what
declaring it says — so this checks them all: Venue.behaviour_info(:callbacks) minus
child_spec/1/start_link/1 (never called here; starting a real process from inside
this suite is a risk no assertion should take) minus exactly two exemptions, named
below. It used to check only a fixed list of eleven callback names that take credentials
as their own first positional argument — a callback that reads a credential out of
opts instead (get_option_chain/2, get_news/1, get_corporate_events/1 and
quantization/1 are the shape a venue that signs every request actually uses this for)
was invisible to that list no matter how it answered with no credential. If your venue
declares :required and any active endpoint takes its credential through opts rather
than as an argument, this now reaches it: stripped_credential_args/2 passes
[credentials: %{}] for every opts position, not merely an opts list that never
carried the key at all.
Two things it deliberately does not do:
It does not run at all unless you declare
:required. A venue where credentials are:no_differenceor:higher_ceilingmay legitimately serve some of these endpoints without one, and this assertion would otherwise be inventing a rule your venue never claimed.It excludes exactly two callbacks from the gate, by name, even on a
:requiredvenue:test_connection/2andget_rate_limit_status/2. Both documentcredentials() | nilon purpose —test_connection/2's whole job is answering "the credential, IF GIVEN, is accepted" — so both are expected to answer plain reachability with none at all. This is a two-name exemption list stated in the assertion's own code, not a judgement call your venue's test file makes — if your fake has another endpoint that is genuinely credential-free by design (a purely local computation that never calls the venue at all, for instance), this assertion will fail on it, and that failure is the finding: either yourcredential_benefit: :requireddeclaration overstates your venue, or that endpoint's exemption belongs argued in your own package's review, not assumed silently.dp_exchange_webullanddp_exchange_robinhood'smarket_status/1were exactly this case as of the widening above, and both are now resolved — differently, because the two venues turned out not to share a reason:market_status/1is not exempt by name, the way the two callbacks above are. Its own callback doc makes an unconditional claim — "crypto venues answer:open" — butdp_exchange_schwabserves equities and its realmarket_status/1calls an authenticated/marketsendpoint; its fake correctly refuses without a credential, and a name-based exemption would have silenced that protection on the one venue where this assertion is doing real work, purely to accommodate two venues where it currently is not.- Instead there is a second, narrower exemption, scoped by what the doc's claim is
actually about: skipped only when your venue's own
asset_classes/0is exactly[:crypto]. Crypto has no exchange-mandated trading session for a credential to gate, sodp_exchange_robinhood(crypto-only) is exempt on that ground — its{:ok, :open}with no credential is the documented behaviour for a crypto-only venue, argued in its own review and recorded inAdapterContract's "17. credential gate" comment rather than assumed silently. dp_exchange_webullis not crypto-only (asset_classes/0is[:crypto, :equity, :option, :future, :event_contract]), so it does not qualify and stays fully gated. It satisfies the gate by declaringmarket_status/1:unsupported({:error, :not_supported}) instead: its OpenAPI documents no market-status or trading-calendar endpoint at all — the one such endpoint Webull publishes anywhere belongs to a separate Broker API product on a different host, reachable only with a broker-tier credential this contract'scredentials()does not model.
See
DpExchange.Core.Venue's ownmarket_status/1doc andAdapterContract's "17. credential gate" comment for the full argument.dp_exchange_webull'sget_fees/2is a second, later case, and it needed a third mechanism rather than reusing either of the first two: it is not exempt by name (a:credentialed-style list, correctly retired for rotting), and it is not exempt by asset class (get_fees/2answers a flat crypto spread even though this venue is not crypto-only, somarket_status_crypto_exempt?/2's ground does not apply). What actually distinguishes it is a fact about the ENDPOINT, not the venue: it answers a rate captured from Webull's own published pricing and never builds a request, so no credential could change what it returns.Capabilities.no_venue_contactnames that fact directly — a list of{name, arity}yourcapabilities/0declares, the same per-endpoint shapeendpointsalready uses, so it cannot rot into one more hand-maintained name list. Declare an endpoint there only when you can point at its real implementation and show the absence of any request-building call; assertion 17 trusts the declaration, so a wrong one defeats the same protection a wrongcredential_benefitwould. SeeCapabilities's own moduledoc for the full argument.It does not assert that your fake's refusal has the same shape as your real venue's (
{:error, {:missing_credentials, :your_venue}}vs whatever your fake returns) — only that it is not{:ok, _}. Matching shapes would need to call the real venue with stripped credentials too, and that is a live network call this suite will not make on your behalf.
If this fails, the fix is in your fake: make it check credentials — wherever your real
facade actually reads them, positional argument or opts[:credentials] — the way your
real Rest/Auth module does, not in this package.
Assertion 18 — a process that links a child it starts must trap exits
Found live in four of five venue packages on 2026-09-07: Feed.init/1 never called
Process.flag(:trap_exit, true), and Socket.start_link/1 ran from inside a Feed
callback — which links the socket to Feed itself, not to a supervisor. An abnormal
socket exit was therefore untrappable and killed Feed, and the Supervisor restarted
it from its static start opts — every subscribe/2 a consumer had made since boot,
gone in the same instant. One socket dying anywhere took the whole feed's subscription
state with it.
Assertion 18 is static, not a behavioural "start the tree and kill a linked child"
test — that was the first design and it was rejected, because starting a venue's real
(non-fake) tree is not reliably network-free. dp_exchange_schwab's Feed dials its
Streamer unconditionally from init/1's own {:continue, :connect}, regardless of
whether anything has ever been subscribed, and the only way around that from Core is a
venue-specific injection option name this suite is expressly forbidden from knowing (see
this file's own "never declare transport"). A static check never starts a process at
all, so it cannot dial out for any venue, present or future — see
DpExchange.Core.LinkSafetyCheck's moduledoc for the full reasoning.
It scans every module in your lib/ that declares GenServer, :gen_statem,
GenStateMachine or WebSockex as a @behaviour, and for each one asks two questions
of its own compiled code, taken as a whole rather than function by function: does it
create a link — a remote call named start_link (any target module, any arity, since
X.start_link always links by OTP convention), Process.link/1, or spawn_link — and
does it also call Process.flag(:trap_exit, true) somewhere. The first without the
second is the violation: a module that manufactures a link to a process it started and
has no way to survive that process dying abnormally.
start_link/1, start_link/2, child_spec/1 and child_spec/2 are never scanned as
the source of a link — every process-behaviour module's own start_link/N delegates to
that behaviour's own start_link (WebSockex.start_link/4, GenServer.start_link/3) to
bootstrap itself, and that link belongs to whoever calls it, not to the module being
bootstrapped. This is the same exclusion assertion 16 already makes for the same
functions, reused rather than reinvented.
What it does not check: whether you handle the resulting {:EXIT, pid, reason}
correctly — clearing coverage, firing a :link_down notice, reopening the child. A
process that traps exits but defines no matching handle_info/2 still survives (use GenServer injects a default that logs and continues), which is the literal claim this
assertion makes and no more. Whether your feed recovers usefully is genuinely different
per venue and is exactly the kind of mechanism this contract's own rule ("never declare
transport") forbids an assertion from encoding — that part is on you, and the five real
fixes (dp_exchange_coinbase e77b542, dp_exchange_gemini 66acd3b,
dp_exchange_webull d0c54a8, dp_exchange_schwab 90dddc6, dp_exchange_robinhood
51ad189) are the reference for what a good recovery looks like.
If this fails, add Process.flag(:trap_exit, true) as the first thing your process's
init/1 does, and a handle_info({:EXIT, pid, reason}, state) clause that isolates the
crash to whatever it actually broke.
Assertion 19 — a struct holding a secret must redact it under inspect/1
Found live in four of five venue packages on 2026-09-07: Feed/Socket held
:credentials as a bare map for their whole lifetime, and OTP's default crash report
prints a process's state in full on termination — a plain map prints every key it holds,
secrets included. Proven by crashing an equivalent process holding %{api_key: "...", api_secret: "..."} as a bare state field and reading the log back. A second leak was
found the same way: a FunctionClauseError's stacktrace prints the actual arguments a
failed clause was called with, so a bad call handed the same raw map to a function whose
every clause failed to match printed it too. Process.flag(:sensitive, true) does not
help — it changes what :sys.get_state/1 and :dbg can see, not how a crash report or a
stacktrace is formatted.
If your package holds a credential anywhere for longer than one function call, wrap it in
a dedicated struct — %YourVenue.Credentials{} — the moment it enters that long-lived
process, and derive Inspect with except: naming every secret field:
defmodule DpExchange.YourVenue.Credentials do
@derive {Inspect, except: [:api_key, :api_secret]}
defstruct [:api_key, :api_secret]
endFour venues shipped exactly this fix independently: dp_exchange_coinbase 4d00669,
dp_exchange_webull 80eaf02, dp_exchange_schwab 336cbd8, dp_exchange_robinhood
cfc4861. dp_exchange_gemini needed none — it signs and discards inside stateless
pipelines and never holds a credential in process state.
Assertion 19 checks this behaviourally, not by looking for @derive in your source.
For every struct your lib/ defines, if any field is named one of fifteen known secret
names (api_key, api_secret, app_key, app_secret, secret, password,
passphrase, token, access_token, refresh_token, client_secret, private_key,
signature, authorization, bearer — see DpExchange.Core.CredentialRedactionCheck's
moduledoc for why each name is on the list, and why client_id deliberately is not), it
builds a real instance of your struct with a distinctive value in that field and searches
the actual inspect/1 output for it. A hand-written defimpl Inspect, for: YourStruct
that never mentions @derive at all passes exactly as validly as the derived form above —
this checks what your struct prints, never how you made it print that way.
Read this as a floor, not a ceiling. A struct field named something this list does not cover but that genuinely carries a secret — your own venue may have a field this family has not seen yet — should still be wrapped and redacted; assertion 19 not flagging it is not permission to leave it in cleartext, only a statement of what this particular check happens to look for today.
Be honest about what this does not catch. The original defect was a raw map, not a
struct — Feed/Socket, pre-fix, held Keyword.get(opts, :credentials) directly in
state, an opaque value never constructed as a literal anywhere in either module's own
compiled code. This assertion locks the struct-based fix in; it cannot and does not reach
back to catch the shape of the bug before that fix existed. If your package holds a
credential as a bare map or keyword list anywhere longer-lived than one function call,
that is a defect this assertion will not find for you. Wrap it in a struct regardless of
whether assertion 19 currently has an opinion about your field names.
If this fails, add @derive {Inspect, except: [...]} (or an equivalent hand-written
defimpl Inspect) naming every secret field your struct holds, following the four
commits above as the reference for the mechanism.
Assertion 20 — subscribe/2 must push a Core.Types.* struct, tagged with runtime_id/0
DpExchange.Core.Venue.subscribe/2's own doc makes an unconditional claim: events arrive
"tagged so a process subscribed to several venues can tell them apart," and "the payload is
a DpExchange.Core.Types.* struct — the same value the pull endpoints return." Nothing
checked either half before this assertion existed. Assertion 16 (internal wiring) catches a
decoder with no caller, but a decoder that is wired and simply never gets called before
the raw response reaches the sink passes every other assertion — the function that forwards
it has a caller, and subscribe/2 still answers :ok.
Assertion 20 calls your Fake's subscribe/2 with @sample_pairs and to: self(), and
checks the first message that arrives: it must be {:dp_exchange, runtime_id, payload}
where runtime_id == your_venue.runtime_id() and payload is a struct under
DpExchange.Core.Types. Fake-only, like assertion 17 — every fake in this family pushes
synchronously inside the call that returns :ok, so nothing here ever dials out.
If this fails, make sure your subscribe/2 builds the same Types.* struct your pull
endpoint for the same data returns — never the venue's raw decoded JSON — and tags every
send with runtime_id(), not a literal atom.
Assertion 21 — a timeframe outside historical_timeframes must be refused
The behavioural half of "Fail closed; never substitute" above, run against your Fake.
Assertion 21 picks a width from Timeframe.nameable/0 that your own
capabilities().historical_timeframes does not name, and calls get_historical_prices/4
with it. The answer must not be {:ok, _} — your fake either errors on the unrecognised
width or (if you gate on credentials first) refuses for that reason instead, either of
which proves the width was never silently served at the nearest one you do have.
A venue that declares the entire Timeframe.nameable/0 vocabulary has nothing left for
this assertion to pick, and the check is a no-op for it — it is not a way to avoid the
check by declaring narrowly, since narrowing what you declare only gives the assertion more
widths to try.
Asset classes are a statement about today
asset_classes says what the package serves now — :crypto, :equity, :option,
:future, :event_contract. It is never a permanent scope boundary, and it must never be
used to justify not implementing something: "this venue's options endpoints are out of scope
because we declared crypto" is the argument in its wrong form, and it has been made in this
family and was wrong.
The only test of scope is does the venue provide it.
Two lists, not one: absence has two causes
Split your :unsupported endpoints into what the venue does not serve and what this package
has not ported, and expose the first through venue_does_not_serve/0.
Both answer a caller identically. Only one of them can ever change, and a host planning around a gap needs to know which it is looking at.
The mislabel goes both ways, and both are defects. A venue's absence filed as a backlog item invents work that cannot be done and quietly implies an endpoint the vendor does not publish. A backlog item filed as the venue's absence hides a capability a consumer could have had. Robinhood shipped four of the first kind and they were found by auditing, not by tests — nothing fails when a comment is wrong.
Every negative gets an audit
Write docs/reference/<venue>/negative-claims.md, tabulating every place your package says
the venue does not do something, with the source and the date you consulted it.
An unverified negative is a substitution exactly like an invented value. "The venue has
no order book" and "we never looked" produce the same {:error, :not_supported}, and only
one of them is true. Across five venues this audit found nine false negatives — every one
of them a working endpoint a consumer was being refused.
Two patterns produced most of them:
- A true statement about one endpoint restated as a claim about the venue. "The stock snapshot does not serve options" is correct; "this venue does not serve options" is not.
- A derived artefact read instead of the vendor. A claim originating in the host application's own adapter, or in a third-party wrapper's README, carried forward until somebody read the vendor's pages.
Dependency floors are a claim exactly like a capability
A ~> X.Y requirement is a claim, in public, that your package runs against every
version from X.Y.0 up. mix.lock never tests that claim — it always resolves the
newest version the requirement allows, so a floor that is actually too low compiles and
passes every ordinary CI run. Only a consumer who resolves an older, still-permitted
version finds out, and finds out as a crash in their application, not a failure in yours.
Four real instances, found in one week, all the same shape:
dp_exchange_webulldeclared{:websockex, "~> 0.4"}while callingWebSockex.send_frame/3, which exists only from 0.5.1. A consumer who resolved 0.4.x got:undef.dp_exchange_webulldeclared{:dp_exchange_core, "~> 0.1.48"}whilecapabilities/0declaresno_venue_contact, aCapabilitiesfield added in Core 0.1.68.Capabilities.new/1builds withstruct!/2, so an older Core raisedKeyErroron everycapabilities/0call — not on the one path that changed, on the one every consumer calls first, at boot.dp_exchange_geminideclared~> 0.1.48while callingTypes.OrderBookDelta.new/1, added in Core 0.1.53.- The first fix for the webull/websockex instance was itself wrong. It raised the
floor to
~> 0.5, reasoning that the thirdsend_frameargument "only exists from 0.5".~> 0.5still permits0.5.0, andsend_frame/3is new in0.5.1— one patch later. The corrected floor was reasoned about, not resolved, and reasoning about a floor is exactly the failure mode this whole section is about. It was only caught becausescript/check_dependency_floor.sh(below) resolved it for real the same day.
A per-API pinning test — function_exported?/3 after Code.ensure_loaded!/1, or a
struct-field check — guards a floor against being lowered later. It cannot catch a floor
that was wrong when it was written, because a test like that has no way to know what
"new" means; it only knows what the author already thought to check. Every instance above
shipped with its code passing whatever tests already existed.
The fix is to resolve the floor, not read it. script/check_dependency_floor.sh
rewrites a scratch copy's mix.exs, pinning every dependency declared without only:
(the ones a consumer actually resolves — never credo, ex_doc, or anything else that
ships only to :dev/:test) to == the exact floor version its own requirement string
names, then runs mix compile --warnings-as-errors plus the package's own
AdapterContract conformance test against that pinned set. Compiling catches an
undefined remote call (the websockex shape); the conformance test — which calls
capabilities/0 under a fully offline Fake — catches a struct or module the floor does
not ship yet (the other two shapes). It runs weekly and on demand, deliberately not on
every push: it freshly resolves the transitive dependencies of whatever it pins (Core's
own req, for instance), which float to whatever is newest today and are not this
package's claim to keep correct, so an unrelated package's release can turn this red for
a reason that has nothing to do with your floor. See the script's own header for the full
reasoning, including why a full mix test is deliberately not what it runs.
Raising mix.lock is never a substitute for raising the floor in mix.exs. Every
instance above passed CI because CI always resolves the newest allowed version — the bug
was invisible from inside this repository by construction. If a change starts calling a
Core function, uses a new Types.* module, or relies on any behaviour a dependency added
after its floor's release, the floor in mix.exs goes up in the same commit, stated as a
version and the reason, the way every floor comment in every venue's mix.exs already
does. Do not wait for the weekly check to say so.
Never do blocking work in a process that owes a reply
This is the failure this family has paid for most often, and every instance looked different until they were lined up:
- #16, #23 — work on the reply path inside a venue's own feed.
- #28 —
Core.PollingFeedran its fetch in a task and then blocked on that task insidehandle_info. The task bounded a hang; it did nothing for the mailbox. A read-onlycoverage/1timed out, the exit propagated out of the venueFeed'shandle_call/3, and the feed died. It restarted without its subscription state, and a live venue went from 61 pairs to 0 and stayed there — alive, idle, passing every liveness probe.
A sweep for the class then found three more, none of which had failed in production yet:
a Streamer bootstrap (a signed HTTP round trip plus a WebSocket connect) inline in
handle_call/3; a whole-catalogue HTTP fetch inline in handle_info/2; and a socket
connect inline in handle_call/3.
The rule
If a callback can block for longer than a caller's timeout, it must not block the
process. Run the slow part in a task, let the result arrive as a message, and defer the
reply with {:noreply, state} + GenServer.reply/2. dp_exchange_webull's
spawn_reconcile/3 is the reference shape; do not invent a second one.
Three things that are easy to get wrong:
- A task that bounds a hang does not unblock a mailbox. Wrapping work in
Task.async/1and then callingTask.yield/2is still blocking — it only bounds how long. That is precisely what #28 was. start_linklinks to its caller. A socket opened inside a task dies when the task exits, moments later. Fetch in the task; connect in the GenServer. Only the fetch is slow enough to matter.- Convert exceptions inside the task.
Task.async/1links, so an unconverted raise arrives as an{:EXIT, …}the GenServer has no clause for — and anything parked waiting on that task is never answered at all.
Reads must carry an explicit timeout too
coverage/1 and status/1 are what a consumer's health check calls. Left on
GenServer.call/2's implicit five seconds while writes name a generous one, any
moment the feed is legitimately busy turns a health check into an exit — and into a
dead consumer process, when the read happens inside the consumer's own handle_call/3.
Every venue in this family now passes @call_timeout on reads as well as writes. It is
the second line of defence, never the fix: a read that has to queue behind something
should wait for it, not die of it.
And the answer must not lie
Where a read cannot reach what it is reporting on, say the least that is true. An empty
coverage map plus a :link_down notice is honest; a remembered coverage map asserts
arrivals nobody confirmed, which is the "nearby substitute where an error belongs" failure
this family keeps finding. "We could not ask" is not "nothing arrived", and only the notice
tells a consumer which one they are looking at.
If your vendor publishes an index, diff it
Across five vendors, a changelog diff caught nothing and an index diff was the only mechanism that ever fired. Three instances now, all real:
- A rate-limit table on
developer.webull.comthat had been published for weeks while this family declared a ceiling five times too permissive, on a venue whose documented penalty is a temporary IP block. - A WebSocket channel on
developer.gemini.comwithdrawn with no changelog entry — the same venue that once withdrew an entire market-data API the same way. - Coinbase's rate-limit pages, previously recorded as "could not be located", sitting in the
vendor's own
sitemap.xmlthe whole time.
So every venue package carries script/check_endpoint_inventory.sh, weekly and
non-blocking, comparing what the vendor says it serves against a committed record. What it
compares depends on what the vendor offers, in this order of preference:
- A machine-readable specification — OpenAPI, AsyncAPI. Diff the operation and channel lists. This is the strongest form; only Gemini publishes one today.
- A sitemap whose pages are one-per-endpoint — diff the set of those paths. Coinbase and Webull are this shape.
- Nothing fetchable — Schwab answers
403to an anonymous reader. There is no automated form; its specification is committed to the repository and re-capture is a human signing in. That is a distinct class, not a degraded one.
Two rules that make the difference between a check and a rubber stamp:
- Fix the claim before you update the record. A difference means a claim this package makes may now be false. Updating the committed inventory first, so the check goes green, destroys the only evidence that anything changed.
- Diff the specification, never the rendered page. An operation list is structured and every entry means something. The pages around it carry build hashes and rotating banners, and a content diff on those is red every week for reasons that are never the reason you care about.
Absent from the documentation is not absent from the venue
When something vanishes, what you have established is that the vendor stopped publishing it — not that the venue stopped serving it. Gemini has diverged from its own documentation in both directions: a socket URL it still published and no longer served, and candle widths it served before it documented them.
So a withdrawal is a reason to label a claim, not automatically to delete it. Deleting asserts a new negative ("this venue does not have X"), and an unverified negative is a substitution exactly like an invented value. Where the thing is private and probing it needs a credential the repository must never hold, labelled is the most honest state available.