macula_direct_dial (macula v13.4.0)
View SourceDirect-dial resolve-and-call: shared internals for macula_request/macula_response, macula_streamer/ macula_stream_sink, and macula_feeder/macula_download.
Not a public API on its own — macula_request:start_link_direct/6,7,8, macula_response:advertise_direct/6,7, macula_stream_sink:start_link_direct/5,6, macula_streamer:advertise_direct/6,7, macula_download:start_link_direct/4,5, and macula_feeder:start_link_direct/5,6 are the entry points. Factored out because RPC, streaming, and content-download all need the same shape of resolve sequence (find_records -> verify -> read the record -> build a quic:// dial URL). Streaming and RPC share the IDENTICAL discovery mechanism — a procedure_advertisement does not distinguish RPC from streaming, only the eventual dial (call_station/7 vs call_stream_station/7) does — so publish_advertisement/4,5 is reused as-is by both providers, and call/6/call_stream/6 share the same candidate resolution and "Trust model" below. Content has no publish step here at all — see "Content" further down.
Resolution
A call starts from a HEAD START when it can. The pool remembers the station that last answered a procedure, and hands it back as a candidate to try before the DHT is asked at all, but only while two things hold: the advertisement it was built from has not reached the end of the lifetime it was remembered with, and the pool STILL HOLDS A LIVE LINK to that station. The live link is what makes this safe to do without a station_endpoint lookup: a link either exists or it does not, so it cannot be stale the way a signed record up to five minutes old can.
It is a head start and never a substitute. The remembered candidate goes through the same trust-independent machinery as any other, gets the same share of the same deadline, and when it fails resolution carries on into the DHT passes exactly as it would have without one. Nothing is skipped except a lookup. What it cannot cover is an advertisement SUPERSEDED by one naming a different station while the remembered one is still inside its own lifetime: the call then goes out to a station that no longer serves the procedure and its answer is returned, where an uncached call would have found the new station. That window is bounded by the remembered lifetime and nothing else, deliberately, because a CALL that has already gone out must not be sent again somewhere else (see macula_station_link:not_sent/1).
Every advertisement that passes trust filtering is a candidate, in the order the DHT returns them. A candidate whose station_endpoint can't be resolved, or whose link doesn't connect ({error, not_connected} from call_station/7 or call_stream_station/7), is passed over for the next one, but only before the request is sent: once a CALL or a stream has gone out, its outcome is returned as it is. When no candidate qualifies, or every one failed before sending, resolution asks the DHT again after a pause that doubles from 100 ms to at most 1 s, and tries a candidate that already failed again only when its advertisement or its station_endpoint record has changed. A record just published on the provider's station has not necessarily replicated to the caller's station yet, so a miss is not final until the deadline. One deadline bounds all of it: each DHT lookup, each candidate's endpoint lookup and connect wait (within a share of the time that remains, at least one second while that much remains), and the request. At the deadline the result is, in this order, the most recent candidate's failure, why the latest answered DHT lookup found nothing qualifying, the latest failed lookup's error, or {error, {unresolved, timeout}}. A failed lookup is retried like an empty pass, and one the deadline cuts off records nothing.
Trust model
Two independent checks, both mandatory, cover what the QUIC/TLS layer cannot. (1) Every candidate procedure_advertisement arrives verified under the node's crypto profile (macula:find_records/2), and is trusted only when it advertises the resolved procedure in the resolved realm and its provider authorization verifies (macula_record:verify_authorization/3, D25 item 6): a procedure with an org namespace needs an authorization for that org, and a procedure without one carries none. Otherwise any node able to sign SOME record could name a real, legitimate station as the server for a procedure it has no authority over, and the station_endpoint check below would still pass (it only proves we reached the station we were told to reach, not that whoever told us so was authorized to). (2) The resolved station_endpoint must be signed by the station itself (station_signed_endpoint/2). The actual QUIC dial proves only that the station holds the key of the self-signed ML-DSA-87 certificate it presents (macula_quic:connect/4), which says nothing of who it is: that is enforced at the application layer, via the cryptographically signed CONNECT/HELLO handshake (the peer identity binding in macula_peering_conn) checked against the exact node_id the signed DHT chain above resolved.
An authorization verifies against the realm key the pool pinned for the call's realm when it started (macula:connect/2's realm_trust => #{RealmId => RealmKey}): the realm key as carried, for the org directory and the procedure delegation, the only authorization form. Without a key pinned for the realm, an advertisement for an org namespaced procedure is never trusted. A realm key never arrives with a request: realm_trust on a call, like the 10.x options verify_cert_chain and cert_chain, is refused by name, with {error, {removed_option, Key}} (see removed_option/2).
A caller checks the authorization from the advertisement alone and looks up no tombstone. A delegation its org withdraws is honoured until it expires, so the caller-side revocation bound is the delegation's maximum lifetime: 30 minutes (D32, macula_record's PROCEDURE_DELEGATION_MAX_LIFETIME_MS), plus 5 minutes of clock tolerance. The realm reissues every 10 minutes, and a revoked provider gets no fresh one.
Content
Content is fetched from the node that shares it (D27): see macula_content_fetch, which resolves a serving station's endpoint with resolve_station_endpoint/3 and opens its stream with macula:call_stream_station/7, and checks no advertisement's authorization: content is verified by its content id.
Dial I/O
The DHT lookups and dials a call runs on come from the dial_io in its Opts, or else from macula. A given dial_io has every function the call runs on, each at the arity its key takes, and may carry the other dial_io() functions; any other is refused with function_clause, in the caller. The option is for tests and for embedding direct dial; other callers leave it out.
Summary
Functions
As call/6 with no options.
Resolve Procedure's provider and call it there directly. Same return shape as macula:call/5; resolve failures surface as {error, {unresolved, Reason}} so a caller can tell "nobody has advertised this via direct-dial yet" apart from a real call failure; once a candidate has failed before the CALL was sent, the most recent candidate's failure is the result instead, such as {error, not_connected}. TimeoutMs bounds resolution, each candidate's connect wait and the CALL itself (see "Resolution" in the module doc). An org namespaced procedure's authorization is checked against the realm key the pool pinned for Realm: see the module doc's "Trust model" section.
As call_stream/6 with no options.
As call/6, but opens a stream (macula:call_stream_station/7's shape) instead of making a single-reply call, built on the exact same resolve+trust machinery — see the module doc. StreamOpts is forwarded to call_stream_station/7 alongside the resolved trust override (mode, owner, etc); its dial_timeout_ms (default 10_000, from 1 to 600_000 as a call's timeout) bounds resolution and each candidate's connect wait, and the stream itself keeps its own deadline. Opts takes no option: realm_trust and verify_cert_chain in it are refused as call/6 refuses them.
As providers/5 with no options.
Who provides Procedure in Realm: every advertisement that passes the trust check a call applies (see "Trust model"), as the provider that signed it and the station it names, in the order the DHT answered. ONE lookup, bounded by TimeoutMs, and no retry: a provider whose record has not replicated to this pool's stations yet is simply not listed. {error, {unresolved, procedure_not_advertised}} when the DHT has no advertisement, {error, {unresolved, no_trusted_advertisement}} when none passes the trust check, {error, {unresolved, Reason}} when the lookup failed. A provider advertising through two stations is listed once per station. Opts takes only dial_io (see "Dial I/O").
As publish_advertisement/5 with no provider authorization.
Publish a signed procedure_advertisement for Procedure, naming Pool's currently-connected station as the serving station. NodeIdentity signs it, and its node_id is the advertiser: it must be the node identity key Pool was started with, since a caller targets that node_id and the station knows the pool's connection by it. Opts may include authorization, the provider authorization an org namespaced procedure needs (D25 item 6), as #{org_directory => Wire, procedure_delegation => Wire}, ttl_ms, and stations, which makes the serving station the first of those node ids the pool is connected to (macula:advertise/5 registers there). cert_chain, a 10.x option authorization replaces, is refused with {error, {removed_option, cert_chain}} before anything is read or put.
The first option in Opts that 11.0.0 removed from a call or an advertisement, as {removed_option, Key}, or none. On a call, the realm keys the pool pins replace verify_cert_chain and realm_trust; on an advertisement, authorization replaces cert_chain.
Whether a call asks for its seal report (DESIGN_E2E_SEAL_REPORT): report is true or false (the default), and any other value is {error, {invalid_option, report}}, before anything is looked up or sent. macula:call_station/8 checks its own report with it too.
As resolve_station_endpoint/3, within 10 seconds.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record, verifying the record's signer is exactly Station and asking again past an absent, expired or malformed record, or a failed lookup, until TimeoutMs has passed — the same discipline call/6 applies once it has a procedure's serving_station. The error is, in this order, the latest answered lookup's own reason (station_endpoint_not_found for no record at all, station_endpoint_expired for one refused as stale, or the malformed record's reason), a failed lookup's own reason, or timeout. An absent record and an expired one are reported apart: the first says the station published no endpoint, the second that it published one and the caller's own clock check refused it.
As resolve_station_endpoint/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
A stream reports through macula:stream_report/1, so report on its open means nothing and is {error, {invalid_option, report}} whatever its value, as the C ABI refuses it, rather than accepted and ignored. macula:call_stream_station/7 checks its own options with it too.
Types
-type dial_io() :: #{links => fun((macula:pool()) -> {ok, [map()]} | {error, term()}), put_record => fun((macula:pool(), map()) -> ok | {error, term()}), find_records => fun((macula:pool(), binary(), pos_integer()) -> {ok, [map()]} | {error, term()}), find_record => fun((macula:pool(), binary(), pos_integer()) -> {ok, map()} | {error, term()}), call_station => fun((macula:pool(), macula_client:seed(), <<_:256>>, macula:realm(), macula:procedure(), term(), pos_integer(), map()) -> {ok, term()} | {ok, term(), macula_station_link:report()} | {error, term()}), call_stream_station => fun((macula:pool(), macula_client:seed(), <<_:256>>, macula:realm(), macula:procedure(), term(), map()) -> {ok, macula:stream()} | {error, term()}), resolved_candidate => fun((macula:pool(), macula:realm(), macula:procedure()) -> {ok, map(), macula_client:seed()} | none), remember_resolved => fun((macula:pool(), macula:realm(), macula:procedure(), map(), non_neg_integer()) -> ok)}.
Functions
-spec call(macula:pool(), macula:realm(), macula:procedure(), term(), 1..600000) -> {ok, term()} | {error, term()}.
As call/6 with no options.
-spec call(macula:pool(), macula:realm(), macula:procedure(), term(), 1..600000, map()) -> {ok, term()} | {ok, term(), macula_station_link:report()} | {error, term()}.
Resolve Procedure's provider and call it there directly. Same return shape as macula:call/5; resolve failures surface as {error, {unresolved, Reason}} so a caller can tell "nobody has advertised this via direct-dial yet" apart from a real call failure; once a candidate has failed before the CALL was sent, the most recent candidate's failure is the result instead, such as {error, not_connected}. TimeoutMs bounds resolution, each candidate's connect wait and the CALL itself (see "Resolution" in the module doc). An org namespaced procedure's authorization is checked against the realm key the pool pinned for Realm: see the module doc's "Trust model" section.
Opts takes provider, a provider's node_id: the call then goes to THAT provider only. Resolution and the trust check are exactly as without it; only candidates whose advertisement that provider signed are tried, the pool's remembered head start included, and a provider with no trusted advertisement by the deadline is {error, {unresolved, provider_not_advertised}}. A caller that wants one answer from each of several providers makes one call per provider (see providers/4). A provider that is not a 32-byte node_id is {error, {invalid_option, provider}}, and realm_trust and verify_cert_chain are refused with {error, {removed_option, Key}}, both before anything is looked up. confidential is preferred (the default) or required; off is refused as {error, {confidentiality, off_needs_explicit_target}} and any other value as {error, {invalid_option, confidential}}, also before anything is looked up. ucan_token (bytes) is presented to every station call, as macula:call_station/8 presents it; any other value is {error, {invalid_option, ucan_token}}, before anything is looked up.
-spec call_stream(macula:pool(), macula:realm(), macula:procedure(), term(), map()) -> {ok, macula:stream()} | {error, term()}.
As call_stream/6 with no options.
-spec call_stream(macula:pool(), macula:realm(), macula:procedure(), term(), map(), map()) -> {ok, macula:stream()} | {error, term()}.
As call/6, but opens a stream (macula:call_stream_station/7's shape) instead of making a single-reply call, built on the exact same resolve+trust machinery — see the module doc. StreamOpts is forwarded to call_stream_station/7 alongside the resolved trust override (mode, owner, etc); its dial_timeout_ms (default 10_000, from 1 to 600_000 as a call's timeout) bounds resolution and each candidate's connect wait, and the stream itself keeps its own deadline. Opts takes no option: realm_trust and verify_cert_chain in it are refused as call/6 refuses them.
-spec providers(macula:pool(), macula:realm(), macula:procedure(), pos_integer()) -> {ok, [#{provider := <<_:256>>, station := <<_:256>>}]} | {error, term()}.
As providers/5 with no options.
-spec providers(macula:pool(), macula:realm(), macula:procedure(), pos_integer(), map()) -> {ok, [#{provider := <<_:256>>, station := <<_:256>>}]} | {error, term()}.
Who provides Procedure in Realm: every advertisement that passes the trust check a call applies (see "Trust model"), as the provider that signed it and the station it names, in the order the DHT answered. ONE lookup, bounded by TimeoutMs, and no retry: a provider whose record has not replicated to this pool's stations yet is simply not listed. {error, {unresolved, procedure_not_advertised}} when the DHT has no advertisement, {error, {unresolved, no_trusted_advertisement}} when none passes the trust check, {error, {unresolved, Reason}} when the lookup failed. A provider advertising through two stations is listed once per station. Opts takes only dial_io (see "Dial I/O").
-spec publish_advertisement(macula:pool(), macula:realm(), macula:procedure(), macula_node_keys:node_key()) -> ok | {error, term()}.
As publish_advertisement/5 with no provider authorization.
-spec publish_advertisement(macula:pool(), macula:realm(), macula:procedure(), macula_node_keys:node_key(), map()) -> ok | {error, term()}.
Publish a signed procedure_advertisement for Procedure, naming Pool's currently-connected station as the serving station. NodeIdentity signs it, and its node_id is the advertiser: it must be the node identity key Pool was started with, since a caller targets that node_id and the station knows the pool's connection by it. Opts may include authorization, the provider authorization an org namespaced procedure needs (D25 item 6), as #{org_directory => Wire, procedure_delegation => Wire}, ttl_ms, and stations, which makes the serving station the first of those node ids the pool is connected to (macula:advertise/5 registers there). cert_chain, a 10.x option authorization replaces, is refused with {error, {removed_option, cert_chain}} before anything is read or put.
The first option in Opts that 11.0.0 removed from a call or an advertisement, as {removed_option, Key}, or none. On a call, the realm keys the pool pins replace verify_cert_chain and realm_trust; on an advertisement, authorization replaces cert_chain.
Whether a call asks for its seal report (DESIGN_E2E_SEAL_REPORT): report is true or false (the default), and any other value is {error, {invalid_option, report}}, before anything is looked up or sent. macula:call_station/8 checks its own report with it too.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>) -> {ok, binary()} | {error, term()}.
As resolve_station_endpoint/3, within 10 seconds.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>, pos_integer()) -> {ok, binary()} | {error, term()}.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record, verifying the record's signer is exactly Station and asking again past an absent, expired or malformed record, or a failed lookup, until TimeoutMs has passed — the same discipline call/6 applies once it has a procedure's serving_station. The error is, in this order, the latest answered lookup's own reason (station_endpoint_not_found for no record at all, station_endpoint_expired for one refused as stale, or the malformed record's reason), a failed lookup's own reason, or timeout. An absent record and an expired one are reported apart: the first says the station published no endpoint, the second that it published one and the caller's own clock check refused it.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>, pos_integer(), map()) -> {ok, binary()} | {error, term()}.
As resolve_station_endpoint/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
-spec stream_report_option(map()) -> {ok, none} | {error, {invalid_option, report}}.
A stream reports through macula:stream_report/1, so report on its open means nothing and is {error, {invalid_option, report}} whatever its value, as the C ABI refuses it, rather than accepted and ignored. macula:call_stream_station/7 checks its own options with it too.