Link Component Deep Dive

The Link is the client-facing interface component that manages transaction coordination and serves as the entry point for all client operations in the Bedrock system.

Overview

The Link acts as the intermediary between client applications and the distributed data plane components. It abstracts the complexity of the distributed system while providing clients with a simple transaction interface.

Location: lib/bedrock/cluster/link.ex

Embedded Distributed Architecture

The Link represents a fundamental shift in how distributed databases interface with applications. Rather than acting as a network service that clients connect to remotely, the Link embeds directly within application processes, providing local transaction coordination while participating in global distributed protocols.

Local-First Transaction Processing

The Link implements Bedrock's core embedded distributed principle: processing within application boundaries rather than across network boundaries. When applications initiate transactions, they're not making network calls to remote database servers—instead, they're invoking local Link processes that coordinate distributed operations on their behalf.

This local-first approach transforms transaction semantics. Applications experience sub-microsecond transaction initiation times because Link processes are co-located within the same memory space. Resource coordination happens locally, eliminating the network latency that dominates traditional distributed database operations.

The Link's transaction management also enables new programming patterns. Because transaction coordination is local, applications can initiate hundreds or thousands of concurrent transactions with minimal overhead, supporting fine-grained transactional workflows that would be prohibitively expensive in client-server architectures.

Unified Failure Domain Coordination

One of the Link's most important architectural contributions is how it leverages embedded distributed systems' simplified failure scenarios. In traditional distributed databases, applications must handle complex failure modes: what happens when the application is healthy but the database is unavailable? Or when network partitions isolate clients from servers?

The Link eliminates these scenarios by creating unified failure domains. When an application fails, its embedded Link fails with it, ensuring there are no orphaned transactions or leaked resources. This unified failure approach dramatically simplifies both application error handling and distributed system recovery protocols.

The Link's worker advertisement functionality exemplifies this design. Rather than maintaining separate service discovery systems, the Link leverages the fact that it knows directly when local workers start or stop—because they're all part of the same failure domain. This knowledge enables more efficient and reliable cluster coordination than traditional service discovery mechanisms.

Embedded Distributed Advantages

The Link's embedded architecture enables capabilities impossible in client-server systems. Because Link processes share memory space with applications, they can implement zero-copy transaction builders, eliminating serialization overhead for transaction state management. They can also provide application-aware optimizations, such as transaction batching based on application request patterns.

The embedded approach also transforms operational characteristics. The Link ensures that transaction capabilities are always available when applications start—there's no separate database service to connect to or dependency to manage. Applications and their transaction coordination deploy and scale together as single units, eliminating the operational complexity of coordinating application and database infrastructure.

This design also enables sophisticated performance optimizations. The Link can learn from application transaction patterns, pre-warm transaction system layouts for frequently used data ranges, and coordinate with local storage workers to optimize data placement for application access patterns.

Core Responsibilities

1. Transaction Lifecycle Management

Transaction Initiation: Creates new Transaction Builder processes for each transaction
Resource Coordination: Manages the lifecycle of transaction-related resources
Client Interface: Provides the primary API surface for client operations

2. Worker Advertisement

Service Discovery: Receives advertisements from new workers joining the cluster
Director Communication: Forwards worker information to the cluster director
Cluster Coordination: Facilitates dynamic cluster membership management

Key APIs

Transaction Management

@spec begin_transaction(link_ref :: ref(), opts :: keyword()) :: 
  {:ok, transaction_pid :: pid()} | {:error, :timeout}

Purpose: Initiates a new transaction by creating a Transaction Builder process.

Process:

Creates a new Transaction Builder process via start_link/1
Passes link reference and transaction system layout to builder
Returns the transaction builder PID for subsequent operations

Usage:

{:ok, transaction_pid} = Link.begin_transaction(link)

Worker Advertisement

@spec advertise_worker(link :: ref(), worker :: pid()) :: :ok

Purpose: Handles dynamic worker registration for cluster membership.

Process:

Receives worker PID from newly started workers
Interrogates worker for capability information
Forwards worker details to cluster director
Always returns :ok (asynchronous operation)

Implementation Details

Server Architecture

The Link uses the Bedrock.Internal.GenServerApi pattern:

API Module: Bedrock.Cluster.Link - Client-facing functions
Server Module: Bedrock.Cluster.Link.Server - GenServer implementation
State Module: Bedrock.Cluster.Link.State - State management

State Management

Key State Components:

%State{
  cluster: Cluster.t(),
  transaction_system_layout: TransactionSystemLayout.t(),
  worker_advertisements: [worker_info],
  # ... other state
}

Integration Points

Upstream Dependencies:

Coordinator: Receives transaction system layout updates
Director: Forwards worker advertisements and receives cluster state

Downstream Dependencies:

Transaction Builder: Creates and manages transaction processes
Sequencer: Coordinates read version assignments through builders

Performance Characteristics

Scalability

Stateless Operations: Most operations are stateless for horizontal scaling
Worker Advertisement: Asynchronous processing prevents blocking

Latency Sources

Transaction Creation: Process spawning overhead for new transactions
State Lookups: In-memory state access (minimal impact)

Optimization Strategies

Process Pooling: Could implement transaction builder pooling
State Caching: Cache frequently accessed transaction system layout

Error Handling

Common Error Scenarios

Transaction Creation Failures:

{:error, :timeout} # System overloaded, cannot create transaction builder

Recovery Behavior:

Process Failures: Link process restart requires transaction retry
Network Partitions: Transactions may need to be retried
Overload: New transaction creation may be throttled

Monitoring and Observability

Key Metrics:

Active transaction count
Worker advertisement rate
Transaction creation latency

Telemetry Events:

Transaction lifecycle events
Worker advertisement processing

Configuration

Key Configuration Options:

Transaction Timeout: Maximum transaction lifetime
Worker Advertisement Buffer: Batching for worker advertisements

Testing Considerations

Unit Testing

Mock transaction system layout for isolated testing
Verify worker advertisement forwarding

Integration Testing

Test with real Transaction Builders
Test worker advertisement flow with Director

Load Testing

High-frequency transaction creation
Worker advertisement bursts

Future Enhancements

Potential Improvements

Connection Pooling: Manage persistent connections to data plane components
Load Balancing: Intelligent routing of transactions to less loaded components
Caching: Cache transaction system layout and worker information

Monitoring Enhancements

Health Checks: Periodic validation of downstream component availability
Performance Metrics: Detailed latency and throughput tracking
Alert Integration: Proactive alerting on creation failures

Cross-Component Workflow Integration

Transaction Processing Flow Role

The Link serves as the entry point in Bedrock's core transaction processing workflow:

Link → Transaction Builder → Commit Proxy → Resolver → Sequencer → Storage

Workflow Context:

Transaction Initiation: Client requests arrive at Link, which creates dedicated Transaction Builder processes
Resource Management: Link manages transaction lifecycle and ensures clean resource cleanup
Error Propagation: Transaction failures propagate back through Link to clients

For the complete transaction flow, see Transaction Processing Deep Dive.

Service Registration Workflow Role

The Link participates in the service discovery and registration workflow:

Foreman → Link → Coordinator → Director