macula_direct_dial (macula v12.2.0)
View SourceDirect-dial resolve-and-call: shared internals for macula_request/macula_response, macula_streamer/ macula_stream_sink, and macula_feeder/macula_download.
Not a public API on its own — macula_request:start_link_direct/6,7,8, macula_response:advertise_direct/6,7, macula_stream_sink:start_link_direct/5,6, macula_streamer:advertise_direct/6,7, macula_download:start_link_direct/4,5, and macula_feeder:start_link_direct/5,6 are the entry points. Factored out because RPC, streaming, and content-download all need the same shape of resolve sequence (find_records -> verify -> read the record -> build a quic:// dial URL). Streaming and RPC share the IDENTICAL discovery mechanism — a procedure_advertisement does not distinguish RPC from streaming, only the eventual dial (call_station/7 vs call_stream_station/7) does — so publish_advertisement/4,5 is reused as-is by both providers, and call/6/call_stream/6 share the same candidate resolution and "Trust model" below. Content has no publish step here at all — see "Content" further down.
Resolution
A call starts from a HEAD START when it can. The pool remembers the station that last answered a procedure, and hands it back as a candidate to try before the DHT is asked at all, but only while two things hold: the advertisement it was built from has not reached the end of the lifetime it was remembered with, and the pool STILL HOLDS A LIVE LINK to that station. The live link is what makes this safe to do without a station_endpoint lookup: a link either exists or it does not, so it cannot be stale the way a signed record up to five minutes old can.
It is a head start and never a substitute. The remembered candidate goes through the same trust-independent machinery as any other, gets the same share of the same deadline, and when it fails resolution carries on into the DHT passes exactly as it would have without one. Nothing is skipped except a lookup. What it cannot cover is an advertisement SUPERSEDED by one naming a different station while the remembered one is still inside its own lifetime: the call then goes out to a station that no longer serves the procedure and its answer is returned, where an uncached call would have found the new station. That window is bounded by the remembered lifetime and nothing else, deliberately, because a CALL that has already gone out must not be sent again somewhere else (see macula_station_link:not_sent/1).
Every advertisement that passes trust filtering is a candidate, in the order the DHT returns them. A candidate whose station_endpoint can't be resolved, or whose link doesn't connect ({error, not_connected} from call_station/7 or call_stream_station/7), is passed over for the next one, but only before the request is sent: once a CALL or a stream has gone out, its outcome is returned as it is. When no candidate qualifies, or every one failed before sending, resolution asks the DHT again after a pause that doubles from 100 ms to at most 1 s, and tries a candidate that already failed again only when its advertisement or its station_endpoint record has changed. A record just published on the provider's station has not necessarily replicated to the caller's station yet, so a miss is not final until the deadline. One deadline bounds all of it: each DHT lookup, each candidate's endpoint lookup and connect wait (within a share of the time that remains, at least one second while that much remains), and the request. At the deadline the result is, in this order, the most recent candidate's failure, why the latest answered DHT lookup found nothing qualifying, the latest failed lookup's error, or {error, {unresolved, timeout}}. A failed lookup is retried like an empty pass, and one the deadline cuts off records nothing.
Trust model
Two independent checks, both mandatory, cover what the QUIC/TLS layer cannot. (1) Every candidate procedure_advertisement arrives verified under the node's crypto profile (macula:find_records/2), and is trusted only when it advertises the resolved procedure in the resolved realm and its provider authorization verifies (macula_record:verify_authorization/3, D25 item 6): a procedure with an org namespace needs an authorization for that org, and a procedure without one carries none. Otherwise any node able to sign SOME record could name a real, legitimate station as the server for a procedure it has no authority over, and the station_endpoint check below would still pass (it only proves we reached the station we were told to reach, not that whoever told us so was authorized to). (2) The resolved station_endpoint must be signed by the station itself (station_signed_endpoint/2). The actual QUIC dial proves only that the station holds the key of the self-signed ML-DSA-87 certificate it presents (macula_quic:connect/4), which says nothing of who it is: that is enforced at the application layer, via the cryptographically signed CONNECT/HELLO handshake (the peer identity binding in macula_peering_conn) checked against the exact node_id the signed DHT chain above resolved.
An authorization verifies against the realm key the pool pinned for the call's realm when it started (macula:connect/2's realm_trust => #{RealmId => RealmKey}): the realm key as carried, for the org directory and the procedure delegation, the only authorization form. Without a key pinned for the realm, an advertisement for an org namespaced procedure is never trusted. A realm key never arrives with a request: realm_trust on a call, like the 10.x options verify_cert_chain and cert_chain, is refused by name, with {error, {removed_option, Key}} (see removed_option/2).
A caller checks the authorization from the advertisement alone and looks up no tombstone. A delegation its org withdraws is honoured until it expires, so the caller-side revocation bound is the delegation's maximum lifetime, six hours (macula_record's REALM_AND_ORG_MAX_LIFETIME_MS), and it lengthens if that lifetime does.
Content
put_content/4 has no resolve step at all — unlike a GET, a PUT names its OWN target: the caller already knows (or is choosing) which station to seed, so it takes Station directly and resolves only that station's own station_endpoint (resolve_station_endpoint/2,3, the same machinery call/6 uses internally for serving_station).
get_content/3 and fetch_content/4 resolve and fetch deliberately WITHOUT the authorization check above: content's threat model genuinely differs from RPC's. An RPC reply is opaque and unverifiable except by trusting whoever answered, so proving the ADVERTISER is authorized matters. Content is content-addressed, and the fetched bytes are checked against the MCID client-side regardless of which peer served them. Single-block content is re-hashed against the MCID (in macula_content_transfer). For chunked content the fetched manifest is used only if its MCID, recomputed from its canonical fields, is the one requested (macula_manifest:verify_mcid/2); each chunk is then hashed against that manifest, and the reassembled bytes are checked against its size and root hash (macula_manifest:verify/2). A rogue or unauthorized announcer can at most refuse to serve or waste a dial; it cannot make a caller accept content that does not match the MCID it asked for. The same holds for trying the next provider after a fetch that fails, so every announced provider is a candidate the way advertisements are for calls. What still matters, and is still mandatory, is (1)'s analogue for content_announcement: the signer must equal the announcer_node it claims (the check macula:find_content_providers/2 makes too), so an attacker cannot at least misattribute who is claiming to serve what.
Dial I/O
The DHT lookups, dials and transfers a call runs on come from the dial_io in its Opts, or else from macula and macula_content_transfer. A given dial_io has every function the call runs on, each at the arity its key takes, and may carry the other dial_io() functions; any other is refused with function_clause, in the caller. The option is for tests and for embedding direct dial; other callers leave it out.
Summary
Functions
As call/6 with no options.
Resolve Procedure's provider and call it there directly. Same return shape as macula:call/5; resolve failures surface as {error, {unresolved, Reason}} so a caller can tell "nobody has advertised this via direct-dial yet" apart from a real call failure; once a candidate has failed before the CALL was sent, the most recent candidate's failure is the result instead, such as {error, not_connected}. TimeoutMs bounds resolution, each candidate's connect wait and the CALL itself (see "Resolution" in the module doc). An org namespaced procedure's authorization is checked against the realm key the pool pinned for Realm: see the module doc's "Trust model" section.
As call_stream/6 with no options.
As call/6, but opens a stream (macula:call_stream_station/7's shape) instead of making a single-reply call, built on the exact same resolve+trust machinery — see the module doc. StreamOpts is forwarded to call_stream_station/7 alongside the resolved trust override (mode, owner, etc); its dial_timeout_ms (default 10_000, from 1 to 600_000 as a call's timeout) bounds resolution and each candidate's connect wait, and the stream itself keeps its own deadline. Opts takes no option: realm_trust and verify_cert_chain in it are refused as call/6 refuses them.
Fetch MCID from the first of its announced providers whose fetch succeeds, within TimeoutMs, by the rules call/6 resolves by (see "Resolution" in the module doc): a provider whose fetch fails, including one whose bytes don't verify against MCID, is passed over for the next, and one that failed is tried again only when its announcement has changed. Fetch(Endpoint, Pinned, ConnectMs, RemainingMs) runs one fetch: Pinned is the dial trust override that pins the announcer, ConnectMs the time that provider gets to connect, RemainingMs what remains of the deadline; it returns {ok, Bytes} or {error, Reason}. macula_download supplies its own, so a cancel reaches whichever transfer is running.
As fetch_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
Fetch MCID from one of its providers, resolved via their signed content_announcements, and dialed directly. Same return shape as macula:get_content/2; resolve failures surface as {error, {unresolved, Reason}}, and once a provider's fetch has failed, the most recent failure is the result instead. TimeoutMs bounds the whole fetch: lookups, each provider's connect wait and the transfers. See fetch_content/4 for how providers are chosen, and the module doc's "Content" section for why it checks no provider authorization, unlike call/6. Only chunked content is discoverable this way — see macula:find_content_providers/2.
As get_content/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
As providers/5 with no options.
Who provides Procedure in Realm: every advertisement that passes the trust check a call applies (see "Trust model"), as the provider that signed it and the station it names, in the order the DHT answered. ONE lookup, bounded by TimeoutMs, and no retry: a provider whose record has not replicated to this pool's stations yet is simply not listed. {error, {unresolved, procedure_not_advertised}} when the DHT has no advertisement, {error, {unresolved, no_trusted_advertisement}} when none passes the trust check, {error, {unresolved, Reason}} when the lookup failed. A provider advertising through two stations is listed once per station. Opts takes only dial_io (see "Dial I/O").
As publish_advertisement/5 with no provider authorization.
Publish a signed procedure_advertisement for Procedure, naming Pool's currently-connected station as the serving station. NodeIdentity signs it, and its node_id is the advertiser: it must be the node identity key Pool was started with, since a caller targets that node_id and the station knows the pool's connection by it. Opts may include authorization, the provider authorization an org namespaced procedure needs (D25 item 6), as #{org_directory => Wire, procedure_delegation => Wire}, and ttl_ms. cert_chain, a 10.x option authorization replaces, is refused with {error, {removed_option, cert_chain}} before anything is read or put.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record and put Bytes there directly. Same return shape as macula:put_content/2; resolve failures surface as {error, {unresolved, Reason}}. TimeoutMs bounds the endpoint lookup and the connect wait (macula:put_content_station/5); the underlying block/manifest transfer has its own internal timeouts.
As put_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
The first option in Opts that 11.0.0 removed from a call or an advertisement, as {removed_option, Key}, or none. On a call, the realm keys the pool pins replace verify_cert_chain and realm_trust; on an advertisement, authorization replaces cert_chain.
Resolve MCID's provider via its signed content_announcement, asking the DHT again past a not-yet-replicated announcement for up to 10 seconds. Returns the announcement of the first provider that qualifies, {error, content_not_announced} when none has by then, or the last lookup's own error when that lookup failed. Deprecated: removed in 11.0.0. Use fetch_content/4, which also moves on to the next provider when a fetch fails.
As resolve_station_endpoint/3, within 10 seconds.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record, verifying the record's signer is exactly Station and asking again past an absent, expired or malformed record, or a failed lookup, until TimeoutMs has passed — the same discipline call/6 applies once it has a procedure's serving_station. The error is, in this order, the latest answered lookup's own reason (station_endpoint_not_found for no record at all, station_endpoint_expired for one refused as stale, or the malformed record's reason), a failed lookup's own reason, or timeout. An absent record and an expired one are reported apart: the first says the station published no endpoint, the second that it published one and the caller's own clock check refused it.
As resolve_station_endpoint/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
Types
-type dial_io() :: #{links => fun((macula:pool()) -> {ok, [map()]} | {error, term()}), put_record => fun((macula:pool(), map()) -> ok | {error, term()}), find_records => fun((macula:pool(), binary(), pos_integer()) -> {ok, [map()]} | {error, term()}), find_record => fun((macula:pool(), binary(), pos_integer()) -> {ok, map()} | {error, term()}), call_station => fun((macula:pool(), macula_client:seed(), <<_:256>>, macula:realm(), macula:procedure(), term(), pos_integer(), map()) -> {ok, term()} | {error, term()}), call_stream_station => fun((macula:pool(), macula_client:seed(), <<_:256>>, macula:realm(), macula:procedure(), term(), map()) -> {ok, macula:stream()} | {error, term()}), put_content_station => fun((macula:pool(), macula_client:seed(), binary(), pos_integer(), map()) -> {ok, macula:mcid()} | {error, term()}), start_get_station => fun((macula:pool(), macula_client:seed(), macula:mcid(), pos_integer(), map()) -> {ok, pid()}), await => fun((pid(), timeout()) -> {ok, term()} | {error, term()}), cancel => fun((pid()) -> ok), resolved_candidate => fun((macula:pool(), macula:realm(), macula:procedure()) -> {ok, map(), macula_client:seed()} | none), remember_resolved => fun((macula:pool(), macula:realm(), macula:procedure(), map(), non_neg_integer()) -> ok)}.
Functions
-spec call(macula:pool(), macula:realm(), macula:procedure(), term(), 1..600000) -> {ok, term()} | {error, term()}.
As call/6 with no options.
-spec call(macula:pool(), macula:realm(), macula:procedure(), term(), 1..600000, map()) -> {ok, term()} | {error, term()}.
Resolve Procedure's provider and call it there directly. Same return shape as macula:call/5; resolve failures surface as {error, {unresolved, Reason}} so a caller can tell "nobody has advertised this via direct-dial yet" apart from a real call failure; once a candidate has failed before the CALL was sent, the most recent candidate's failure is the result instead, such as {error, not_connected}. TimeoutMs bounds resolution, each candidate's connect wait and the CALL itself (see "Resolution" in the module doc). An org namespaced procedure's authorization is checked against the realm key the pool pinned for Realm: see the module doc's "Trust model" section.
Opts takes provider, a provider's node_id: the call then goes to THAT provider only. Resolution and the trust check are exactly as without it; only candidates whose advertisement that provider signed are tried, the pool's remembered head start included, and a provider with no trusted advertisement by the deadline is {error, {unresolved, provider_not_advertised}}. A caller that wants one answer from each of several providers makes one call per provider (see providers/4). A provider that is not a 32-byte node_id is {error, {invalid_option, provider}}, and realm_trust and verify_cert_chain are refused with {error, {removed_option, Key}}, both before anything is looked up.
-spec call_stream(macula:pool(), macula:realm(), macula:procedure(), term(), map()) -> {ok, macula:stream()} | {error, term()}.
As call_stream/6 with no options.
-spec call_stream(macula:pool(), macula:realm(), macula:procedure(), term(), map(), map()) -> {ok, macula:stream()} | {error, term()}.
As call/6, but opens a stream (macula:call_stream_station/7's shape) instead of making a single-reply call, built on the exact same resolve+trust machinery — see the module doc. StreamOpts is forwarded to call_stream_station/7 alongside the resolved trust override (mode, owner, etc); its dial_timeout_ms (default 10_000, from 1 to 600_000 as a call's timeout) bounds resolution and each candidate's connect wait, and the stream itself keeps its own deadline. Opts takes no option: realm_trust and verify_cert_chain in it are refused as call/6 refuses them.
-spec fetch_content(macula:pool(), macula:mcid(), pos_integer(), fun((binary(), map(), pos_integer(), pos_integer()) -> {ok, binary()} | {error, term()})) -> {ok, binary()} | {error, term()}.
Fetch MCID from the first of its announced providers whose fetch succeeds, within TimeoutMs, by the rules call/6 resolves by (see "Resolution" in the module doc): a provider whose fetch fails, including one whose bytes don't verify against MCID, is passed over for the next, and one that failed is tried again only when its announcement has changed. Fetch(Endpoint, Pinned, ConnectMs, RemainingMs) runs one fetch: Pinned is the dial trust override that pins the announcer, ConnectMs the time that provider gets to connect, RemainingMs what remains of the deadline; it returns {ok, Bytes} or {error, Reason}. macula_download supplies its own, so a cancel reaches whichever transfer is running.
-spec fetch_content(macula:pool(), macula:mcid(), pos_integer(), fun((binary(), map(), pos_integer(), pos_integer()) -> {ok, binary()} | {error, term()}), map()) -> {ok, binary()} | {error, term()}.
As fetch_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
-spec get_content(macula:pool(), macula:mcid(), pos_integer()) -> {ok, binary()} | {error, term()}.
Fetch MCID from one of its providers, resolved via their signed content_announcements, and dialed directly. Same return shape as macula:get_content/2; resolve failures surface as {error, {unresolved, Reason}}, and once a provider's fetch has failed, the most recent failure is the result instead. TimeoutMs bounds the whole fetch: lookups, each provider's connect wait and the transfers. See fetch_content/4 for how providers are chosen, and the module doc's "Content" section for why it checks no provider authorization, unlike call/6. Only chunked content is discoverable this way — see macula:find_content_providers/2.
-spec get_content(macula:pool(), macula:mcid(), pos_integer(), map()) -> {ok, binary()} | {error, term()}.
As get_content/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
-spec providers(macula:pool(), macula:realm(), macula:procedure(), pos_integer()) -> {ok, [#{provider := <<_:256>>, station := <<_:256>>}]} | {error, term()}.
As providers/5 with no options.
-spec providers(macula:pool(), macula:realm(), macula:procedure(), pos_integer(), map()) -> {ok, [#{provider := <<_:256>>, station := <<_:256>>}]} | {error, term()}.
Who provides Procedure in Realm: every advertisement that passes the trust check a call applies (see "Trust model"), as the provider that signed it and the station it names, in the order the DHT answered. ONE lookup, bounded by TimeoutMs, and no retry: a provider whose record has not replicated to this pool's stations yet is simply not listed. {error, {unresolved, procedure_not_advertised}} when the DHT has no advertisement, {error, {unresolved, no_trusted_advertisement}} when none passes the trust check, {error, {unresolved, Reason}} when the lookup failed. A provider advertising through two stations is listed once per station. Opts takes only dial_io (see "Dial I/O").
-spec publish_advertisement(macula:pool(), macula:realm(), macula:procedure(), macula_node_keys:node_key()) -> ok | {error, term()}.
As publish_advertisement/5 with no provider authorization.
-spec publish_advertisement(macula:pool(), macula:realm(), macula:procedure(), macula_node_keys:node_key(), map()) -> ok | {error, term()}.
Publish a signed procedure_advertisement for Procedure, naming Pool's currently-connected station as the serving station. NodeIdentity signs it, and its node_id is the advertiser: it must be the node identity key Pool was started with, since a caller targets that node_id and the station knows the pool's connection by it. Opts may include authorization, the provider authorization an org namespaced procedure needs (D25 item 6), as #{org_directory => Wire, procedure_delegation => Wire}, and ttl_ms. cert_chain, a 10.x option authorization replaces, is refused with {error, {removed_option, cert_chain}} before anything is read or put.
-spec put_content(macula:pool(), <<_:256>>, binary(), pos_integer()) -> {ok, macula:mcid()} | {error, term()}.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record and put Bytes there directly. Same return shape as macula:put_content/2; resolve failures surface as {error, {unresolved, Reason}}. TimeoutMs bounds the endpoint lookup and the connect wait (macula:put_content_station/5); the underlying block/manifest transfer has its own internal timeouts.
-spec put_content(macula:pool(), <<_:256>>, binary(), pos_integer(), map()) -> {ok, macula:mcid()} | {error, term()}.
As put_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
The first option in Opts that 11.0.0 removed from a call or an advertisement, as {removed_option, Key}, or none. On a call, the realm keys the pool pins replace verify_cert_chain and realm_trust; on an advertisement, authorization replaces cert_chain.
-spec resolve_content_provider(macula:pool(), macula:mcid()) -> {ok, map()} | {error, term()}.
Resolve MCID's provider via its signed content_announcement, asking the DHT again past a not-yet-replicated announcement for up to 10 seconds. Returns the announcement of the first provider that qualifies, {error, content_not_announced} when none has by then, or the last lookup's own error when that lookup failed. Deprecated: removed in 11.0.0. Use fetch_content/4, which also moves on to the next provider when a fetch fails.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>) -> {ok, binary()} | {error, term()}.
As resolve_station_endpoint/3, within 10 seconds.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>, pos_integer()) -> {ok, binary()} | {error, term()}.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record, verifying the record's signer is exactly Station and asking again past an absent, expired or malformed record, or a failed lookup, until TimeoutMs has passed — the same discipline call/6 applies once it has a procedure's serving_station. The error is, in this order, the latest answered lookup's own reason (station_endpoint_not_found for no record at all, station_endpoint_expired for one refused as stale, or the malformed record's reason), a failed lookup's own reason, or timeout. An absent record and an expired one are reported apart: the first says the station published no endpoint, the second that it published one and the caller's own clock check refused it.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>, pos_integer(), map()) -> {ok, binary()} | {error, term()}.
As resolve_station_endpoint/3, on the dial_io in Opts (see "Dial I/O" in the module doc).