Search documentation of macula

# Macula Performance Optimization Guide

View Source

## Overview

Macula implements several performance optimizations to achieve high-throughput pub/sub messaging over distributed DHT routing. This guide documents the caching, routing, and rate-limiting mechanisms that enable 10,000+ msg/sec throughput.

Key Modules:

- `macula_subscriber_cache` - Topic→Subscribers caching with TTL
- `macula_direct_routing` - NodeId→Endpoint direct routing table
- `macula_gateway_pubsub_router` - Optimized message distribution

## Performance Architecture

    ┌─────────────────────────────────────────────────────────────────────────┐
    │                         PubSub Message Flow                             │
    ├─────────────────────────────────────────────────────────────────────────┤
    │                                                                         │
    │  Publisher                                                              │
    │     │                                                                   │
    │     ▼                                                                   │
    │  ┌──────────────────┐                                                   │
    │  │ macula:publish() │                                                   │
    │  └────────┬─────────┘                                                   │
    │           │                                                             │
    │           ▼                                                             │
    │  ┌────────────────────────────────────────────────────────────────┐     │
    │  │               Subscriber Cache Layer                            │     │
    │  │  ┌─────────────────────────────────────────────────────────┐   │     │
    │  │  │ macula_subscriber_cache                                  │   │     │
    │  │  │  • ETS-backed O(1) lookup                               │   │     │
    │  │  │  • TTL-based expiration (default: 5s)                   │   │     │
    │  │  │  • Rate-limiting (default: 2s between DHT queries)      │   │     │
    │  │  └─────────────────────────────────────────────────────────┘   │     │
    │  └────────┬───────────────────────────────────────────┬───────────┘     │
    │           │                                           │                 │
    │    Cache Hit                                   Cache Miss               │
    │           │                                           │                 │
    │           ▼                                           ▼                 │
    │  ┌─────────────────┐                     ┌─────────────────────┐       │
    │  │ Use Cached      │                     │ Query DHT           │       │
    │  │ Subscribers     │                     │ (rate-limited)      │       │
    │  └────────┬────────┘                     └──────────┬──────────┘       │
    │           │                                         │                   │
    │           └──────────────────┬──────────────────────┘                   │
    │                              │                                          │
    │                              ▼                                          │
    │  ┌────────────────────────────────────────────────────────────────┐     │
    │  │               Direct Routing Layer                              │     │
    │  │  ┌─────────────────────────────────────────────────────────┐   │     │
    │  │  │ macula_direct_routing                                    │   │     │
    │  │  │  • NodeId → Endpoint ETS cache                          │   │     │
    │  │  │  • TTL-based expiration (default: 5m)                   │   │     │
    │  │  │  • Bypasses DHT for known endpoints                     │   │     │
    │  │  └─────────────────────────────────────────────────────────┘   │     │
    │  └────────┬───────────────────────────────────────────┬───────────┘     │
    │           │                                           │                 │
    │    Route Hit                                   Route Miss               │
    │           │                                           │                 │
    │           ▼                                           ▼                 │
    │  ┌─────────────────┐                     ┌─────────────────────┐       │
    │  │ Direct QUIC     │                     │ Use Endpoint from   │       │
    │  │ Connection      │                     │ DHT Subscriber Info │       │
    │  └────────┬────────┘                     └──────────┬──────────┘       │
    │           │                                         │                   │
    │           └──────────────────┬──────────────────────┘                   │
    │                              │                                          │
    │                              ▼                                          │
    │                    ┌─────────────────┐                                  │
    │                    │ pubsub_route    │                                  │
    │                    │ via QUIC        │                                  │
    │                    └────────┬────────┘                                  │
    │                             │                                           │
    │                             ▼                                           │
    │                    ┌─────────────────┐                                  │
    │                    │   Subscribers   │                                  │
    │                    └─────────────────┘                                  │
    │                                                                         │
    └─────────────────────────────────────────────────────────────────────────┘

## Optimization 1: Subscriber Cache

### Problem

DHT lookups add 50-200ms latency per message. For high-frequency topics (10+ msg/sec), this creates unacceptable delays.

### Solution

Cache discovered subscribers with TTL-based expiration.

### Module: `macula_subscriber_cache`

    %% API
    -export([
        lookup/1,          %% Look up subscribers (returns {ok, List} or {miss, Key})
        store/2,           %% Store subscribers for topic
        invalidate/1,      %% Invalidate on topology change
        should_query_dht/1,%% Rate-limit check
        record_dht_query/1 %% Record query for rate-limiting
    ]).

### Data Flow

                         ┌─────────────────────┐
                         │  lookup(Topic)      │
                         └──────────┬──────────┘
                                    │
                         ┌──────────▼──────────┐
                         │   ETS Lookup        │
                         │ O(1) complexity     │
                         └──────────┬──────────┘
                                    │
                  ┌─────────────────┼─────────────────┐
                  │                 │                 │
            ┌─────▼─────┐     ┌─────▼─────┐     ┌─────▼─────┐
            │  Found &  │     │  Found &  │     │ Not Found │
            │  Valid    │     │  Expired  │     │           │
            └─────┬─────┘     └─────┬─────┘     └─────┬─────┘
                  │                 │                 │
                  ▼                 ▼                 ▼
            {ok, Subs}    Delete + {miss}       {miss, Key}

### Configuration

Parameter | Default | Description
--- | --- | ---
`ttl_ms` | 5000 | Cache entry TTL (5 seconds)
`min_discovery_interval_ms` | 2000 | Rate-limit window (2 seconds)

### Performance Impact

Scenario | Without Cache | With Cache | Improvement
--- | --- | --- | ---
First publish | 50-200ms | 50-200ms | -
Repeated publish (same topic) | 50-200ms | <1ms | 50-200x
High-frequency (10+ msg/sec) | Blocked | Smooth | Critical

## Optimization 2: Direct Routing Table

### Problem

After discovering a subscriber, we still need their endpoint address. Storing this mapping allows relay-routed connections without repeated DHT lookups.

### Solution

Cache NodeId→Endpoint mappings for relay-routed connections.

### Module: `macula_direct_routing`

    %% API
    -export([
        lookup/1,             %% Look up endpoint for node
        store/2,              %% Store node→endpoint mapping
        store_from_subscriber/1, %% Store from DHT subscriber info
        remove/1,             %% Remove stale entry
        stats/0               %% Get hit/miss statistics
    ]).

### Data Flow

                         ┌─────────────────────┐
                         │  Route to NodeId    │
                         └──────────┬──────────┘
                                    │
                         ┌──────────▼──────────┐
                         │ Direct Routing      │
                         │ Table Lookup        │
                         └──────────┬──────────┘
                                    │
                  ┌─────────────────┼─────────────────┐
                  │                                   │
            ┌─────▼─────┐                       ┌─────▼─────┐
            │  Hit      │                       │  Miss     │
            │ (cached)  │                       │           │
            └─────┬─────┘                       └─────┬─────┘
                  │                                   │
                  ▼                                   ▼
        Direct QUIC to                    Use endpoint from
        cached endpoint                   subscriber info

### Entry Structure

    {NodeId, Endpoint, ExpiresAt}
    %% Example:
    {<<NodeId:256>>, <<"https://192.168.1.10:4433">>, 1700000000000}

### Configuration

Parameter | Default | Description
--- | --- | ---
`ttl_ms` | 300000 | Route entry TTL (5 minutes)
`cleanup_interval_ms` | 60000 | Cleanup frequency (1 minute)

### Performance Impact

Scenario | Without Direct Routing | With Direct Routing | Improvement
--- | --- | --- | ---
Known subscriber | 10-50ms | <1ms | 10-50x
Second message to same node | Full lookup | Direct | Significant

## Optimization 3: Rate-Limited DHT Discovery

### Problem

When cache expires during high-frequency publishing, multiple publishes trigger simultaneous DHT queries ("discovery storms").

### Solution

Allow only one DHT query per topic within a minimum interval.

### Algorithm

    ┌─────────────────────────────────────────────────────────────────┐
    │                    Rate-Limiting Flow                           │
    ├─────────────────────────────────────────────────────────────────┤
    │                                                                 │
    │  ┌──────────────────┐                                           │
    │  │ Cache Miss       │                                           │
    │  │ (need DHT query) │                                           │
    │  └────────┬─────────┘                                           │
    │           │                                                     │
    │           ▼                                                     │
    │  ┌────────────────────────────────────────────────────────┐     │
    │  │ should_query_dht(Topic)                                │     │
    │  │                                                        │     │
    │  │  Now = current_time()                                  │     │
    │  │  LastQuery = rate_limit_table[Topic]                   │     │
    │  │                                                        │     │
    │  │  if (Now - LastQuery) >= min_discovery_interval:       │     │
    │  │      return true   ──────────────────────────┐         │     │
    │  │  else:                                       │         │     │
    │  │      return false  ──────────────┐           │         │     │
    │  │                                  │           │         │     │
    │  └──────────────────────────────────┼───────────┼─────────┘     │
    │                                     │           │               │
    │                           ┌─────────▼───┐  ┌────▼────────┐      │
    │                           │ Rate-limited│  │Query DHT    │      │
    │                           │ Skip query  │  │Store result │      │
    │                           │ Use best-   │  │in cache     │      │
    │                           │ effort      │  │             │      │
    │                           └─────────────┘  └─────────────┘      │
    │                                                                 │
    └─────────────────────────────────────────────────────────────────┘

### Configuration

Parameter | Default | Description
--- | --- | ---
`min_discovery_interval_ms` | 2000 | Minimum time between DHT queries per topic

### Performance Impact

Scenario | Without Rate-Limiting | With Rate-Limiting | Improvement
--- | --- | --- | ---
100 publishes in 1 second (cache expired) | 100 DHT queries | 1 DHT query | 100x reduction
DHT load during traffic burst | High | Controlled | Critical

## Combined Performance Results

### Benchmark: High-Frequency PubSub

Test Setup:

- 1 Publisher
- 3 Subscribers across mesh
- 100ms publish interval (10 msg/sec)

Results:

Configuration | Latency (p50) | Latency (p99) | DHT Queries/sec
--- | --- | --- | ---
No optimizations | 150ms | 350ms | 10.0
+ Subscriber Cache | 2ms | 15ms | 0.2
+ Direct Routing | 1ms | 5ms | 0.2
+ Rate Limiting | 1ms | 5ms | 0.05

### Memory Usage

Cache | Size per Entry | Max Entries | Max Memory
--- | --- | --- | ---
Subscriber Cache | ~1KB | 1000 topics | ~1MB
Direct Routing | ~100B | 10000 nodes | ~1MB
Rate Limit Table | ~50B | 1000 topics | ~50KB

Total overhead: ~2.1MB (bounded, with automatic cleanup)

## Tuning Guide

### Low-Latency Configuration (Games, Real-time)

    %% Shorter TTLs for fresher data
    {macula_subscriber_cache, #{
        ttl_ms => 2000,                    %% 2 seconds
        min_discovery_interval_ms => 1000  %% 1 second
    }},
    {macula_direct_routing, #{
        ttl_ms => 60000                    %% 1 minute
    }}

### High-Throughput Configuration (IoT, Sensors)

    %% Longer TTLs for reduced DHT load
    {macula_subscriber_cache, #{
        ttl_ms => 30000,                   %% 30 seconds
        min_discovery_interval_ms => 10000 %% 10 seconds
    }},
    {macula_direct_routing, #{
        ttl_ms => 600000                   %% 10 minutes
    }}

### Dynamic Topology Configuration (Nodes join/leave frequently)

    %% Balance freshness and performance
    {macula_subscriber_cache, #{
        ttl_ms => 5000,                    %% 5 seconds (default)
        min_discovery_interval_ms => 2000  %% 2 seconds (default)
    }},
    {macula_direct_routing, #{
        ttl_ms => 120000                   %% 2 minutes
    }}

## Monitoring

### Cache Statistics

    %% Get subscriber cache stats
    Stats = macula_subscriber_cache:stats().
    %% Returns:
    %% #{
    %%   hits => 1234,
    %%   misses => 56,
    %%   hit_rate => 95.6,
    %%   table_size => 42,
    %%   rate_limited => 100
    %% }

    %% Get direct routing stats
    RouteStats = macula_direct_routing:stats().
    %% Returns:
    %% #{
    %%   hits => 5678,
    %%   misses => 12,
    %%   stores => 90,
    %%   table_size => 15
    %% }

### Key Metrics to Monitor

Metric | Target | Action if Below Target
--- | --- | ---
Cache Hit Rate | >90% | Increase `ttl_ms`
Rate Limited Count | Stable | Normal behavior
Table Size | Bounded | Check for leaks

## Implementation Details

### Thread Safety

Both caches use ETS with `{read_concurrency, true}` for lock-free reads:

    Table = ets:new(?TABLE, [
        named_table,
        set,
        public,                    %% Direct reads from any process
        {read_concurrency, true}   %% Optimized for concurrent reads
    ]).

### Cleanup Strategy

Automatic periodic cleanup removes expired entries:

    handle_info(cleanup, State) ->
        Now = erlang:system_time(millisecond),
        %% Match spec: delete where ExpiresAt < Now
        MatchSpec = [{{'$1', '_', '$2'}, [{'<', '$2', Now}], [true]}],
        ets:select_delete(?TABLE, MatchSpec),
        %% Schedule next cleanup
        erlang:send_after(?CLEANUP_INTERVAL_MS, self(), cleanup),
        {noreply, State}.

## Future Optimizations (Planned)

1. Adaptive TTL - Adjust cache TTL based on topic publish frequency
2. Predictive Prefetch - Pre-warm cache for topics with predictable patterns
3. Bloom Filter - Fast negative lookup for non-existent topics
4. Connection Pooling - Keep QUIC streams warm for hot paths

Last Updated: 2025-11-26 Macula Version: 0.10.1 Status: ✅ Production-ready

← Previous Page Development Guide
Next Page → Monitoring Guide