Search documentation of macula

# Macula

View Source

BEAM-native HTTP/3 mesh networking for distributed applications

## What is Macula?

Macula is an Erlang/OTP library that provides a complete distributed mesh networking stack over HTTP/3 (QUIC). It enables BEAM applications to form relay-assisted mesh networks with:

- Federated relay mesh -- outbound-only connections, no open ports
- Zero configuration clustering via UDP multicast gossip
- NAT-friendly transport using QUIC (single UDP port)
- Erlang distribution over relay -- `net_adm:ping` across firewalls

## Features

### Mesh Networking

Feature | Description
--- | ---
HTTP/3 over QUIC | NAT-friendly, firewall-friendly, built-in TLS 1.3
Kademlia DHT | Decentralized routing with O(log N) lookups
Multi-tenant realms | Isolated namespaces for different applications
Connection pooling | 94.5% hit rate, LRU eviction

### Pub/Sub Messaging

    %% Subscribe to a topic
    {ok, SubRef} = macula:subscribe(Peer, <<"sensors.temperature">>, fun(Msg) ->
        io:format("Received: ~p~n", [Msg])
    end).

    %% Publish to subscribers
    ok = macula:publish(Peer, <<"sensors.temperature">>, #{value => 23.5}).

    %% Unsubscribe when done
    ok = macula:unsubscribe(Peer, SubRef).

### RPC (Request/Response)

    %% Advertise a procedure handler
    {ok, _Ref} = macula:advertise(Peer, <<"math.add">>, fun(#{a := A, b := B}) ->
        {ok, #{result => A + B}}
    end).

    %% Call the procedure (discovers provider via DHT)
    {ok, #{result := 5}} = macula:call(Peer, <<"math.add">>, #{a => 2, b => 3}).

### Gossip Clustering

Zero-configuration cluster formation using UDP multicast:

    %% Start gossip-based clustering
    ok = macula_cluster:start_cluster(#{
        strategy => gossip,
        secret => <<"my_cluster_secret">>  %% Optional HMAC authentication
    }).

    %% Nodes auto-discover via multicast 230.1.1.251:45892

### Erlang Distribution Over Relay Mesh

Full OTP distribution tunneled through the relay mesh — nodes only need outbound connectivity. No VPNs, no open ports.

    %% Enable relay distribution
    os:putenv("MACULA_DIST_MODE", "relay"),
    macula_dist_relay:register_mesh_client(Client),
    macula_dist_relay:advertise_dist_accept(),

    %% Now standard OTP distribution works across firewalls
    net_adm:ping('other@remote-host').  %% => pong
    gen_server:call({Name, 'other@remote-host'}, Request).  %% works

Feature | Status
--- | ---
Handshake over relay | 5-message dist_util handshake through pub/sub
AES-256-GCM encryption | Tunnel bytes encrypted, relay cannot read ETF
Supervised bridges | gen_server per tunnel under simple_one_for_one
Cross-relay tunnels | Nodes on different relays connect via peering
Relay reconnection | Bridge re-acquires client after relay restart
Metrics | Per-tunnel byte/message counters

See Distribution Over Mesh Guide for details.

### Content Transfer (Mesh Artifacts)

Content-addressed storage and transfer for distributing OTP releases across the mesh:

    %% Publish a file to the mesh
    {ok, MCID} = macula_content:publish("/path/to/release.tar.gz").

    %% Fetch from any provider
    {ok, Binary} = macula_content:fetch(MCID).

Feature | Description
--- | ---
MCID | Content-addressed identifiers (BLAKE3/SHA256)
Merkle verification | Chunk-level integrity
Parallel download | From multiple providers
Want/Have/Block | Efficient mesh protocol

### Authorization (DID + UCAN)

Decentralized capability-based authorization:

Component | Purpose
--- | ---
DID | Decentralized identifiers for namespace ownership
UCAN | Capability tokens for delegated permissions
Namespace | `did:macula:io.example.user` owns `io.example.user.*`

### Hierarchical DHT (Bridge System)

Fractal mesh hierarchy with query escalation:

    Cluster → Street → Neighborhood → City → Province → Country → Region → Global

When a DHT query fails locally, it escalates to parent levels with results cached at lower levels.

## Quick Start

### Installation

Add to your `rebar.config`:

    {deps, [
        {macula, "0.42.7"}
    ]}.

Or in Elixir `mix.exs`:

    defp deps do
      [
        {:macula, "~> 0.42.7"}
      ]
    end

### Basic Usage

    %% Start macula application
    application:ensure_all_started(macula).

    %% Connect to the mesh
    {ok, Peer} = macula:connect(<<"quic://seed.example.com:9443">>, #{
        realm => <<"io.example.myapp">>,
        node_id => <<"node-001">>
    }).

    %% Subscribe to events
    {ok, _SubRef} = macula:subscribe(Peer, <<"events.orders">>, fun(Order) ->
        process_order(Order)
    end).

    %% Advertise an RPC procedure
    {ok, _AdvRef} = macula:advertise(Peer, <<"inventory.check">>, fun(#{sku := Sku}) ->
        {ok, check_inventory(Sku)}
    end).

    %% Call an RPC procedure (discovers provider via DHT)
    {ok, Result} = macula:call(Peer, <<"inventory.check">>, #{sku => <<"ABC123">>}).

## Configuration

### Environment Variables

Variable | Default | Description
--- | --- | ---
`MACULA_QUIC_PORT` | `9443` | QUIC listener port
`MACULA_REALM` | `com.example.realm` | Default realm
`HEALTH_PORT` | `8080` | Health check HTTP port
`MACULA_TLS_MODE` | `development` | TLS mode (`production`/`development`)
`CLUSTER_STRATEGY` | `gossip` | Cluster strategy (`gossip`/`static`/`mdns`)
`CLUSTER_SECRET` | - | Shared secret for gossip HMAC

### Application Config

    {macula, [
        {quic_port, 9443},
        {realm, <<"io.example.myapp">>},
        {tls_mode, production},
        {tls_cacertfile, "/etc/ssl/certs/ca-certificates.crt"}
    ]}.

## Documentation

Guide | Description
--- | ---
Distribution Over Mesh | Relay-tunneled Erlang distribution
Cluster API Guide | Clustering and distribution
Content Transfer Guide | Mesh artifact distribution
NAT Traversal Guide | NAT techniques
DHT Guide | Kademlia DHT internals
Authorization Guide | DID/UCAN security
TLS Configuration | Production TLS setup

## Version History

Version | Date | Highlights
--- | --- | ---
v0.42.7 | Apr 2026 | Distribution over relay mesh — cross-relay, encryption, supervision, metrics (48 tests)
v0.40.0 | Apr 2026 | First Erlang dist-over-mesh: net_adm:ping across 3 countries via relay tunnel
v0.19.2 | Jan 2026 | README rework with feature sections and SVGs
v0.19.1 | Jan 2026 | Gossip clustering, static strategy (34 tests)
v0.19.0 | Jan 2026 | Content transfer system, MCID, Want/Have/Block (171 tests)
v0.18.0 | Jan 2026 | Cluster API for bc_gitops integration (19 tests)
v0.16.0 | Dec 2025 | Registry system, Ed25519 signing (60 tests)
v0.15.0 | Dec 2025 | Gossip protocol, CRDT replication (29 tests)
v0.14.0 | Dec 2025 | Masterless CRDT architecture (48 tests)
v0.13.0 | Dec 2025 | Hierarchical DHT, bridge system (40 tests)
v0.12.0 | Nov 2025 | Complete NAT traversal (70 tests)

See CHANGELOG.md for full history.

## Architecture

Macula follows an always-on architecture where every node has all capabilities:

    macula_root (application supervisor)
    ├── macula_gateway_system (QUIC transport + relay handler)
    ├── macula_pubsub_system (pub/sub messaging)
    ├── macula_rpc_system (RPC request/response)
    ├── macula_routing_system (Kademlia DHT)
    ├── macula_bootstrap_system (mesh discovery)
    ├── macula_membership_system (gossip clustering)
    ├── macula_bridge_system (hierarchical mesh escalation)
    ├── macula_platform_system (CRDT coordination)
    ├── macula_registry_system (package distribution)
    ├── macula_content_system (mesh content transfer)
    ├── macula_cert_system (TLS certificate management)
    └── macula_dist_system (Erlang distribution)
        ├── macula_dist_bridge_sup (relay tunnel bridges)
        ├── macula_dist_discovery (DHT node discovery)
        └── macula_cluster_strategy (optional auto-clustering)

## Related Projects

Project | Description
--- | ---
macula-ecosystem | Documentation hub
macula-console | Management console
bc-gitops | GitOps reconciler
macula-tweann | Neuroevolution framework

## Community

- Hex.pm: hex.pm/packages/macula
- GitHub: github.com/macula-io/macula
- Issues: github.com/macula-io/macula/issues

## License

Apache 2.0 - See LICENSE for details.

Built with the BEAM

Next Page → Changelog