macula (macula v10.14.5)
View SourceMacula SDK — Public API for mesh applications.
This is the main entry point for applications using the Macula SDK.
Apps connect via connect/2, which returns a macula_client pool that internally wraps N peering links to N stations. publish/4,5, subscribe/4,5, unsubscribe/2, call/5, advertise/5, unadvertise/3, call_stream/5, advertise_stream/5, and unadvertise_stream/3 route through the pool with realm-per-call semantics. See macula_pubsub for the slice module of the publish/subscribe surface.
LOCAL streaming (call_stream/2,3, open_stream/3,4, advertise_stream/2,3, unadvertise_stream/1) dispatches in-process via macula_stream_local — for unit tests and same-BEAM pairs.
Erlang distribution over the mesh ships via join_mesh/1 (V2 pool carrier) or join_dist_relay/1 (dedicated dist relay). See macula_dist_pool / macula_dist_system.
Summary
Functions
Abort the stream with an error frame.
Advertise a procedure handler on a V2 pool. Fans out to every healthy link and stores in pool state for replay on link respawn. See macula_client:advertise/4.
Advertise a LOCAL in-process streaming procedure (default: server_stream).
Advertise a LOCAL in-process streaming procedure with mode.
Advertise a streaming procedure on a V2 pool. Fans out to every healthy link and stores in pool state for replay on link respawn. See macula_client:advertise_stream/5.
Wait for the terminal reply (client-stream / bidi).
Issue a CALL frame against a V2 pool. First-success across the pool's healthy links. See macula_client:call/5.
Issue a CALL to ONE specific station, dialing it directly even if it is not in the pool's seed set. Station is a seed URL (e.g. <<"quic://[::1]:4433">>). The pool reuses an existing link or dials and monitors a new one, waits for the handshake, and calls there. This is the direct-dial data path: resolve a serving_station and its endpoint, then reach it in one hop. See macula_client:call_station/6.
As call_station/6, presenting a capability token to a gated provider via Opts (#{ucan_token => Token}). Empty/absent = none. Slice 7b dual-trust. Opts also carries the per-call TLS trust override for this dial: verify, expected_node_id, and pin_tls_cert (see macula_client:call_station/8).
Open a LOCAL in-process server-stream call. Used for unit tests and same-BEAM dispatch via macula_stream_local.
Open a LOCAL in-process server-stream call with options.
Open a streaming RPC against a V2 pool. Picks the first currently-healthy link and opens the stream there; the returned stream is sticky-to-link (errors with peer_down if the link dies; caller re-opens). See macula_client:call_stream/5.
Open a streaming RPC by DIALING a specific station directly (direct-dial), instead of routing through an existing pool link — the streaming analogue of call_station/6. Compose it with DHT resolution (find_records -> read_procedure_advertisement -> station_endpoint) to reach a stream provider in one hop, exactly as a unary caller does. Opts may set dial_timeout_ms (default 10_000) and a mode. Opts also carries the per-call TLS trust override for this dial: verify, expected_node_id, pin_tls_cert (see macula_client:call_station/8).
OTP child spec to drop a V2 pool into a caller's supervision tree.
Stop a V2 pool. Every subscriber receives a final {macula_event_gone, SubRef, pool_closed} message.
Half-close the write side; recv still drains.
Connect to the Macula relay mesh and return a pool handle.
Ensure this node is running in distributed mode.
Resolve every host currently announcing an MCID: hosts that stored a chunked put (_content.put_manifest) and got content_announcementd automatically by the station on receipt (macula_content_announcer). get_content/2 already reaches a copy via the connected station's own 1-hop peer relay, so this is for a caller that wants to know WHO holds an MCID, or to dial a specific one directly with get_content_station/4,5 — e.g. when the connected station's relay hop budget does not reach the host (a partial-mesh pair with no mutual peer), or to route around a specific host deliberately.
Fetch a record from the mesh DHT by its macula_record:storage_key/1.
Fetch EVERY record stored at Key — the full multi-value set, e.g. every procedure_advertisement under one procedure's storage key. Where find_record/2 returns the first record (or not_found), this returns the whole list, empty when none.
Return every record of a given type currently visible from the pool's connected stations.
Fetch the bytes for a previously-stored MCID. Returns {error, not_found} if no provider in the pool's reach holds a copy (for chunked content, if any single chunk is unreachable). Dispatches on the MCID's codec byte: 16#55 (raw/single-block) fetches one block, BLAKE3-verified by the station before it leaves the store; 16#56 (manifest) fetches the manifest, then every chunk in order, reassembles, and verifies the whole against the manifest's size and Merkle root before returning.
As get_content/2, dialing Station directly (reusing a live link or dialing + waiting up to TimeoutMs for one) instead of picking from the pool's existing links — the content-transfer counterpart to call_station/6. Station and TimeoutMs mean exactly what they do there; the underlying block/manifest transfer has its own internal timeouts regardless of TimeoutMs, which bounds only the connect wait. See find_content_providers/2 to resolve a station to dial, or macula_direct_dial:get_content/3 to resolve-and-fetch in one call.
As get_content_station/4, with a per-call TLS trust override for this dial — verify, expected_node_id, pin_tls_cert (see call_station/8). See get_content/2 on why a malformed MCID is rejected here rather than reaching macula_content_transfer.
Get the Erlang cluster cookie.
Enable Erlang distribution over a dedicated dist relay (macula-io/macula-dist-relay).
Join the Macula relay mesh with Erlang distribution.
Per-link snapshot of a V2 pool — one entry per spawned link with its peer station node_id (pubkey), dial host, pid, and connected flag. Use this to resolve a specific station (by pubkey or hostname) to its link for targeted, per-station operations. See macula_client:links/1 for the link_info() shape.
Subscribe to node up/down events.
Open a LOCAL in-process client-stream or bidi call. Used for unit tests and same-BEAM dispatch via macula_stream_local.
Open a LOCAL in-process stream with explicit mode.
Publish to (Realm, Topic) on Pool. Equivalent to publish/5 with empty opts.
Publish to (Realm, Topic) on Pool with options. See macula_pubsub:publish/5 for honored opts.
Store Bytes in the mesh's content store and return its MCID (Macula Content ID — 34 bytes: version, codec, then a 32-byte hash). Content that fits in one block (byte_size(Bytes) =< macula_manifest:default_chunk_size/0, 256 KiB) is sent as a single _content.put_block — the MCID is <<1, 16#55, BLAKE3(Bytes)>>, unchanged since v4.2.7. Larger content is split into chunks (macula_manifest:create/1), each chunk sent via its own _content.put_block, then a content_manifest via _content.put_manifest; the returned MCID is the manifest's (<<1, 16#56, _/binary>>), Merkle-rooted over every chunk. Either way the station verifies each block's hash before accepting it.
As put_content/2, dialing Station directly (reusing a live link or dialing + waiting up to TimeoutMs for one) instead of picking from the pool's existing links — the content-transfer counterpart to call_station/6. Station and TimeoutMs mean exactly what they do there; the underlying block/manifest transfer has its own internal timeouts regardless of TimeoutMs, which bounds only the connect wait. See macula_direct_dial:put_content/4 to resolve a station by identity and put in one call.
As put_content_station/4, with a per-call TLS trust override for this dial — verify, expected_node_id, pin_tls_cert (see call_station/8).
Store a signed record in the mesh DHT via a V2 pool.
Receive the next chunk (blocks).
Send a binary chunk on the stream.
Send a chunk with explicit encoding.
Set the Erlang cluster cookie.
Server-side: emit the terminal reply value.
Aggregate health snapshot of a V2 pool. Suitable for /health or /status endpoints; not for hot-loop polling. See macula_client:status/1 for the full shape.
Subscribe Subscriber to (Realm, Topic) on Pool. Equivalent to subscribe/5 with empty opts.
Subscribe Subscriber to (Realm, Topic) on Pool with options. The delivery option chooses how a single publisher's out-of-order arrivals are handled
Subscribe with a callback function. The SDK spawns a small receiver process internally and invokes the callback once per inbound event. See macula_pubsub:subscribe_callback/4.
Subscribe to live record-stored events filtered by type.
Stop advertising a procedure on a V2 pool.
Stop advertising a LOCAL streaming procedure.
Stop advertising a streaming procedure on a V2 pool.
Unsubscribe from node up/down events.
Drop a pool subscription. Idempotent.
Cancel a subscribe_records/3 subscription.
Types
-type m_record() :: macula_record:m_record().
-type mcid() :: <<_:272>>.
-type pool() :: macula_client:pool().
-type procedure() :: binary().
-type realm() :: <<_:256>>.
32-byte realm tag.
-type record_key() :: <<_:256>>.
DHT storage key — macula_record:storage_key/1 output.
-type record_type() :: macula_record:type_tag().
-type stream() :: pid().
-type stream_mode() :: server_stream | client_stream | bidi.
-type topic() :: binary().
Functions
Abort the stream with an error frame.
-spec advertise(pool(), realm(), procedure(), macula_client:handler(), map()) -> ok | {error, term()}.
Advertise a procedure handler on a V2 pool. Fans out to every healthy link and stores in pool state for replay on link respawn. See macula_client:advertise/4.
-spec advertise_stream(procedure(), stream_handler()) -> ok | {error, term()}.
Advertise a LOCAL in-process streaming procedure (default: server_stream).
-spec advertise_stream(procedure(), stream_mode(), stream_handler()) -> ok | {error, term()}.
Advertise a LOCAL in-process streaming procedure with mode.
-spec advertise_stream(pool(), realm(), procedure(), stream_mode(), stream_handler()) -> ok | {error, term()}.
Advertise a streaming procedure on a V2 pool. Fans out to every healthy link and stores in pool state for replay on link respawn. See macula_client:advertise_stream/5.
Wait for the terminal reply (client-stream / bidi).
Issue a CALL frame against a V2 pool. First-success across the pool's healthy links. See macula_client:call/5.
-spec call_station(pool(), macula_client:seed(), realm(), procedure(), term(), pos_integer()) -> {ok, term()} | {error, term()}.
Issue a CALL to ONE specific station, dialing it directly even if it is not in the pool's seed set. Station is a seed URL (e.g. <<"quic://[::1]:4433">>). The pool reuses an existing link or dials and monitors a new one, waits for the handshake, and calls there. This is the direct-dial data path: resolve a serving_station and its endpoint, then reach it in one hop. See macula_client:call_station/6.
-spec call_station(pool(), macula_client:seed(), realm(), procedure(), term(), pos_integer(), map()) -> {ok, term()} | {error, term()}.
As call_station/6, presenting a capability token to a gated provider via Opts (#{ucan_token => Token}). Empty/absent = none. Slice 7b dual-trust. Opts also carries the per-call TLS trust override for this dial: verify, expected_node_id, and pin_tls_cert (see macula_client:call_station/8).
Open a LOCAL in-process server-stream call. Used for unit tests and same-BEAM dispatch via macula_stream_local.
Open a LOCAL in-process server-stream call with options.
Open a streaming RPC against a V2 pool. Picks the first currently-healthy link and opens the stream there; the returned stream is sticky-to-link (errors with peer_down if the link dies; caller re-opens). See macula_client:call_stream/5.
-spec call_stream_station(pool(), macula_client:seed(), realm(), procedure(), term(), map()) -> {ok, stream()} | {error, term()}.
Open a streaming RPC by DIALING a specific station directly (direct-dial), instead of routing through an existing pool link — the streaming analogue of call_station/6. Compose it with DHT resolution (find_records -> read_procedure_advertisement -> station_endpoint) to reach a stream provider in one hop, exactly as a unary caller does. Opts may set dial_timeout_ms (default 10_000) and a mode. Opts also carries the per-call TLS trust override for this dial: verify, expected_node_id, pin_tls_cert (see macula_client:call_station/8).
-spec child_spec(term(), [macula_client:seed()], macula_client:opts()) -> supervisor:child_spec().
OTP child spec to drop a V2 pool into a caller's supervision tree.
-spec close(pool()) -> ok.
Stop a V2 pool. Every subscriber receives a final {macula_event_gone, SubRef, pool_closed} message.
-spec close_send(stream()) -> ok.
Half-close the write side; recv still drains.
-spec close_stream(stream()) -> ok.
Close a V1 stream (both sides). Renamed from close/1 in 3.11.0 because close/1 now refers to the V2 pool surface.
-spec connect([macula_client:seed()], macula_client:opts()) -> {ok, pool()} | {error, term()}.
Connect to the Macula relay mesh and return a pool handle.
Seeds is a list of relay endpoints (URL binaries/strings or #{host, port} maps). The pool spawns one peering link per seed and routes ops with replication, replay, and event dedup. Returns immediately; link handshakes complete asynchronously.
Honored opts (full reference: macula_client:opts()):
identity— pool's Ed25519 keypair; auto-generated if absent.replication_factor— links per PUBLISH (default 1).capabilities— per-link bitfield (default 0).alpn— QUIC ALPN list (default[<<"macula">>]).connect_timeout_ms— per-link CONNECT/HELLO deadline (default 30_000).dedup_window_ms,dedup_sweep_ms— inbound-EVENT dedup tunables.
Legacy opts silently dropped (with a one-shot logger:notice): relays (use the Seeds positional argument), realm (V2 is realm-per-call), site (no V2 analog), connections (one link per seed; add more seeds to grow the pool).
See macula_client for the canonical pool implementation and macula_pubsub for the slice module.
-spec ensure_distributed() -> ok | {error, term()}.
Ensure this node is running in distributed mode.
Resolve every host currently announcing an MCID: hosts that stored a chunked put (_content.put_manifest) and got content_announcementd automatically by the station on receipt (macula_content_announcer). get_content/2 already reaches a copy via the connected station's own 1-hop peer relay, so this is for a caller that wants to know WHO holds an MCID, or to dial a specific one directly with get_content_station/4,5 — e.g. when the connected station's relay hop budget does not reach the host (a partial-mesh pair with no mutual peer), or to route around a specific host deliberately.
Each entry's signature is verified, AND its signer must equal the announcer_node it claims — same discipline as station_endpoint resolution — before its endpoint is trusted; unverifiable, signer-mismatched, or malformed records are dropped, not surfaced as errors. Single-block content (put via _content.put_block alone) is not announced — resolving its MCID returns {ok, []}.
-spec find_record(pool(), record_key()) -> {ok, m_record()} | {error, not_found | term()}.
Fetch a record from the mesh DHT by its macula_record:storage_key/1.
Returns {error, not_found} when no record exists at the key. The returned record's signature should be verified via macula_record:verify/1 before its payload is trusted.
-spec find_records(pool(), record_key()) -> {ok, [m_record()]} | {error, term()}.
Fetch EVERY record stored at Key — the full multi-value set, e.g. every procedure_advertisement under one procedure's storage key. Where find_record/2 returns the first record (or not_found), this returns the whole list, empty when none.
The relay's local store is a signer-deduped multiset: one record per signing key at a storage key, so N providers of one procedure return N records. Each returned record's signature should be verified via macula_record:verify/1 before its payload is trusted.
-spec find_records_by_type(pool(), record_type()) -> {ok, [m_record()]} | {error, term()}.
Return every record of a given type currently visible from the pool's connected stations.
Coverage depends on each station's view of the DHT — a single station sees its local replicas plus whatever its peers have gossiped. Aggregating across the full mesh requires querying multiple stations and deduplicating by record key.
Fetch the bytes for a previously-stored MCID. Returns {error, not_found} if no provider in the pool's reach holds a copy (for chunked content, if any single chunk is unreachable). Dispatches on the MCID's codec byte: 16#55 (raw/single-block) fetches one block, BLAKE3-verified by the station before it leaves the store; 16#56 (manifest) fetches the manifest, then every chunk in order, reassembles, and verifies the whole against the manifest's size and Merkle root before returning.
A thin blocking wrapper over macula_content_transfer:start_get/2 + await/1 — see the note on put_content/2. MCID must carry one of the two codec bytes put_content/2 ever mints (16#55 single-block, 16#56 chunked manifest) — anything else can't have come from this SDK's own put path (a corrupted record, a caller's encoding bug, or hostile input on a path that turns user-controlled bytes into an MCID) and is rejected here rather than reaching macula_content_transfer's internal dispatch, whose is_chunked/2 clauses assume this shape and previously crashed the calling process's linked worker on anything else.
-spec get_content_station(pool(), macula_client:seed(), mcid(), pos_integer()) -> {ok, binary()} | {error, not_found | term()}.
As get_content/2, dialing Station directly (reusing a live link or dialing + waiting up to TimeoutMs for one) instead of picking from the pool's existing links — the content-transfer counterpart to call_station/6. Station and TimeoutMs mean exactly what they do there; the underlying block/manifest transfer has its own internal timeouts regardless of TimeoutMs, which bounds only the connect wait. See find_content_providers/2 to resolve a station to dial, or macula_direct_dial:get_content/3 to resolve-and-fetch in one call.
-spec get_content_station(pool(), macula_client:seed(), mcid(), pos_integer(), map()) -> {ok, binary()} | {error, not_found | invalid_mcid | term()}.
As get_content_station/4, with a per-call TLS trust override for this dial — verify, expected_node_id, pin_tls_cert (see call_station/8). See get_content/2 on why a malformed MCID is rejected here rather than reaching macula_content_transfer.
-spec get_cookie() -> atom().
Get the Erlang cluster cookie.
Enable Erlang distribution over a dedicated dist relay (macula-io/macula-dist-relay).
Different from join_mesh/1: - Connects to a dist relay (port 4434, ALPN macula-dist), NOT the pub/sub station mesh - No mesh_client, no pub/sub subscriptions — only dist traffic - Uses raw QUIC stream routing with no MessagePack overhead
Options: - url (required): <<"quic://relay.example.com:4434">>
After this returns ok, standard OTP distribution (rpc:call/4, gen_server:call/3 across nodes, pg groups, etc.) works across firewalls via the dist relay.
Join the Macula relay mesh with Erlang distribution.
After calling this, standard OTP distribution works across firewalls. Opts takes:
relays(required) — list of seed URLs for the V2 pool.identity— V2 pool'smacula_identity:key_pair(). Default: auto-generated.
Internally builds a V2 macula_client:pool() and registers it with macula_dist_pool as the carrier for _dist.tunnel.* traffic. Dist tunnel frames travel under the all-zeros realm (protocol-internal infrastructure, not bound to any user realm).
-spec links(pool()) -> {ok, [macula_client:link_info()]}.
Per-link snapshot of a V2 pool — one entry per spawned link with its peer station node_id (pubkey), dial host, pid, and connected flag. Use this to resolve a specific station (by pubkey or hostname) to its link for targeted, per-station operations. See macula_client:links/1 for the link_info() shape.
-spec monitor_nodes() -> ok.
Subscribe to node up/down events.
Open a LOCAL in-process client-stream or bidi call. Used for unit tests and same-BEAM dispatch via macula_stream_local.
Open a LOCAL in-process stream with explicit mode.
Publish to (Realm, Topic) on Pool. Equivalent to publish/5 with empty opts.
Publish to (Realm, Topic) on Pool with options. See macula_pubsub:publish/5 for honored opts.
Store Bytes in the mesh's content store and return its MCID (Macula Content ID — 34 bytes: version, codec, then a 32-byte hash). Content that fits in one block (byte_size(Bytes) =< macula_manifest:default_chunk_size/0, 256 KiB) is sent as a single _content.put_block — the MCID is <<1, 16#55, BLAKE3(Bytes)>>, unchanged since v4.2.7. Larger content is split into chunks (macula_manifest:create/1), each chunk sent via its own _content.put_block, then a content_manifest via _content.put_manifest; the returned MCID is the manifest's (<<1, 16#56, _/binary>>), Merkle-rooted over every chunk. Either way the station verifies each block's hash before accepting it.
The whole transfer — every block call plus the manifest call for chunked content — rides one dedicated QUIC stream on one pinned pool link (see PLAN_PER_STREAM_QUIC_ISOLATION.md Phase 2), so a large blob transfer no longer head-of-line-blocks other RPC/PubSub traffic on the same connection.
A thin blocking wrapper over macula_content_transfer:start_put/2 + await/1 — see that module for the addressable form (a live pid, real cancel with a peer-visible abort, pause/resume/multi-stream as later phases land). PLAN_PUSH_UPLOAD.md Phase 1.
-spec put_content_station(pool(), macula_client:seed(), binary(), pos_integer()) -> {ok, mcid()} | {error, term()}.
As put_content/2, dialing Station directly (reusing a live link or dialing + waiting up to TimeoutMs for one) instead of picking from the pool's existing links — the content-transfer counterpart to call_station/6. Station and TimeoutMs mean exactly what they do there; the underlying block/manifest transfer has its own internal timeouts regardless of TimeoutMs, which bounds only the connect wait. See macula_direct_dial:put_content/4 to resolve a station by identity and put in one call.
-spec put_content_station(pool(), macula_client:seed(), binary(), pos_integer(), map()) -> {ok, mcid()} | {error, term()}.
As put_content_station/4, with a per-call TLS trust override for this dial — verify, expected_node_id, pin_tls_cert (see call_station/8).
Store a signed record in the mesh DHT via a V2 pool.
Build the record via the typed constructors in macula_record (node_record/3,4, content_announcement/3,4, tombstone/3,4, realm_directory/3,4, procedure_advertisement/3,4, etc.) then sign it with macula_record:sign/2. The relay validates the signature on receipt; an invalid signature returns {error, bad_signature}. Successful stores propagate to the K-nearest peers in the DHT under the record's macula_record:storage_key/1.
Receive the next chunk (blocks).
Send a binary chunk on the stream.
Send a chunk with explicit encoding.
Set the Erlang cluster cookie.
Server-side: emit the terminal reply value.
-spec status(pool()) -> {ok, macula_client:status()}.
Aggregate health snapshot of a V2 pool. Suitable for /health or /status endpoints; not for hot-loop polling. See macula_client:status/1 for the full shape.
Subscribe Subscriber to (Realm, Topic) on Pool. Equivalent to subscribe/5 with empty opts.
Subscribe Subscriber to (Realm, Topic) on Pool with options. The delivery option chooses how a single publisher's out-of-order arrivals are handled:
ordered(default) — per-publisher FIFO by seq; out-of-order arrivals are buffered and released in order, a genuinely missing seq skipped afterorder_timeout_ms(aconnect/2option, default 250ms).latest_only— deliver only seqs newer than the highest seen for that publisher (drop stale); no buffering, no delay.as_arrives— deliver in raw arrival order; the consumer orders it itself.
-spec subscribe_callback(pool(), realm(), topic(), macula_pubsub:callback()) -> {ok, reference()} | {error, term()}.
Subscribe with a callback function. The SDK spawns a small receiver process internally and invokes the callback once per inbound event. See macula_pubsub:subscribe_callback/4.
-spec subscribe_records(pool(), record_type(), fun((m_record()) -> any())) -> {ok, reference()} | {error, term()}.
Subscribe to live record-stored events filtered by type.
The callback receives each newly-stored record of the given type. Returns a subscription reference for unsubscribe_records/2. Topic shape is _dht.records.<type>.stored, rendered with the type tag as a decimal integer for log friendliness.
Stop advertising a procedure on a V2 pool.
-spec unadvertise_stream(procedure()) -> ok.
Stop advertising a LOCAL streaming procedure.
Stop advertising a streaming procedure on a V2 pool.
-spec unmonitor_nodes() -> ok.
Unsubscribe from node up/down events.
Drop a pool subscription. Idempotent.
Cancel a subscribe_records/3 subscription.