Deployment Guide

This guide covers running FerricStore as a server: native release, Docker, AWS Fargate, Kubernetes, and cluster layouts. For local development, start with Getting Started. For production security, pair this guide with Security.

Beta: FerricStore is currently a 0.x beta. Use exact image/package versions and validate upgrades before critical production use. Compatibility guarantees will harden with the 1.0 release line.

Recommended path:

1. Use Docker for local smoke tests.
2. Use a native release or container image for production.
3. Put data on durable fast storage; use local NVMe for benchmarks.
4. Enable protected mode, ACL, and TLS before exposing the server.

Native Release (Recommended for Benchmarks)

Build and run directly on the host for maximum performance:

MIX_ENV=prod mix release ferricstore
_build/prod/rel/ferricstore/bin/ferricstore start

Environment variables:

BEAM VM Tuning

The release ships with rel/vm.args.eex containing production BEAM flags:

• +P 1048576 -- max processes (headroom for many connections)
• +Q 1048576 -- max ports/file descriptors
• +stbt db -- bind schedulers to CPU cores
• +sbwt very_short -- scheduler busy-wait for lower latency
• +swt very_low -- wake schedulers faster on new work
• +sub true -- scheduler utilization balancing
• +A 128 -- async thread pool for file I/O
• +MBas aobf / +MHas aobf -- binary allocator strategy (address-order best-fit)
• +Muacul 0 -- disable carrier utilization limit

Socket Options

The TCP acceptor uses the following socket options (hardcoded in ferricstore_server):

Docker

Basic

docker run -p 6388:6388 \
  -e FERRICSTORE_PROTECTED_MODE=false \
  -v ferricstore_data:/data \
  quay.io/ferricstore/ferricstore:0.11.8

The official image is published publicly to Quay.io:

docker pull quay.io/ferricstore/ferricstore:0.11.8

Current release images are published as multi-arch images for linux/amd64 and linux/arm64.

Docker Production Notes

For write-heavy workloads, prefer a direct data mount on durable fast storage and make sure the container runtime allows io_uring syscalls.

docker run -p 6388:6388 \
  --security-opt seccomp=unconfined \
  -e FERRICSTORE_PROTECTED_MODE=true \
  -v /mnt/nvme/ferricstore:/data \
  quay.io/ferricstore/ferricstore:0.11.8

Why io_uring Matters

Docker's default seccomp profile blocks the io_uring_setup, io_uring_enter, and io_uring_register syscalls. Without them, FerricStore falls back to synchronous pwrite + fdatasync for Bitcask writes — roughly 2-3x slower for write-heavy workloads.

Options (pick one):

1. seccomp:unconfined — disables all seccomp filtering (simplest)
2. Custom seccomp profile — add only the 3 io_uring syscalls:

{
  "defaultAction": "SCMP_ACT_ALLOW",
  "syscalls": [
    {"names": ["io_uring_setup", "io_uring_enter", "io_uring_register"],
     "action": "SCMP_ACT_ALLOW"}
  ]
}

security_opt:
  - seccomp:./ferricstore-seccomp.json

Why NVMe Direct Mount Matters

Docker's overlay filesystem adds a VFS layer between the application and disk. For a storage engine that does its own caching (ETS) and write-ahead logging (WARaft segment log + Bitcask), this overhead is pure waste.

Mount the NVMe partition directly:

-v /mnt/nvme/ferricstore:/data

Or for maximum IOPS, use a RAM-backed tmpfs (data lost on restart):

docker run --tmpfs /data:size=8g ...

Cluster container examples will be documented after that layout is tested as part of the release process.

AWS Fargate

The repository includes two deployable ECS/Fargate profiles for FerricStore OSS.

The disposable single-task profile is in deploy/aws/fargate. It uses an internal Network Load Balancer, encrypted task-local ephemeral storage mounted at /data, the isolated readiness endpoint, CloudWatch Logs, and ECS Exec.

cd deploy/aws/fargate
cp terraform.tfvars.example terraform.tfvars
terraform init
terraform plan
terraform apply

All data is lost when the task stops or is replaced, so this layout is for disposable development, demos, SDK integration, CI, caches, and similar OSS workloads. EFS is not used because FerricStore's LMDB projection cannot safely run on NFS. Fargate cannot reattach an existing ECS-managed EBS data volume to a replacement task.

The stack keeps the ECS desired count at one and serializes replacements. A service replica count greater than one is not a FerricStore cluster. The stack does not create S3, DynamoDB, EFS, or EBS resources. Use the native-release or Kubernetes local/block-storage layouts for durable workloads. See the stack README for networking, security, cost, and cleanup notes, and the single-task support contract for the exact failure behavior.

The clustered profile is in deploy/aws/fargate-cluster. It runs three stable logical node slots across three availability zones. Each slot has one desired-count-one ECS service and one task-local replica. Cloud Map moves the stable node name when ECS assigns a replacement IP, periodic EPMD discovery reconnects it, and Raft rebuilds one blank replacement from the two survivors.

cd deploy/aws/fargate-cluster
cp terraform.tfvars.example terraform.tfvars
terraform init
terraform plan
terraform apply

Image changes are registered by Terraform but deployed only through scripts/deploy-sequential.sh. It replaces one node and verifies full local catch-up before touching the next. The cluster has no S3, DynamoDB, EFS, or EBS data dependency. It tolerates one task-disk loss; simultaneous loss or upgrade of two replicas is unsupported, and loss of all three loses all data. See the cluster support contract for discovery, health, rollout, and failure details.

Kubernetes

Basic Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ferricstore
spec:
  replicas: 3
  template:
    spec:
      containers:
        - name: ferricstore
          image: quay.io/ferricstore/ferricstore:0.11.8
          ports:
            - name: native
              containerPort: 6388
            - name: health-probe
              containerPort: 6381
          env:
            - name: FERRICSTORE_PROTECTED_MODE
              value: "false"
            - name: FERRICSTORE_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: FERRICSTORE_COOKIE
              valueFrom:
                secretKeyRef:
                  name: ferricstore-secrets
                  key: erlang-cookie
            - name: FERRICSTORE_DISCOVERY
              value: "dns"
            - name: FERRICSTORE_DNS_NAME
              value: "ferricstore-headless"
          volumeMounts:
            - name: data
              mountPath: /data
          livenessProbe:
            httpGet:
              path: /health/live
              port: health-probe
          readinessProbe:
            httpGet:
              path: /health/ready
              port: health-probe
      volumes:
        - name: data
          emptyDir: {}

For cluster deployments, create the ferricstore-secrets Secret with a strong shared cookie before starting pods:

kubectl create secret generic ferricstore-secrets \
  --from-literal=erlang-cookie="$(openssl rand -base64 32)"

Use DNS discovery in Kubernetes. Gossip discovery is loopback-bound by default; only set FERRICSTORE_GOSSIP_IF_ADDR/FERRICSTORE_GOSSIP_MULTICAST_IF when you intentionally want multicast gossip on a private pod/node network and have firewalled FERRICSTORE_GOSSIP_PORT.

Optimized for Production

Enable io_uring

Kubernetes uses the container runtime's seccomp profile. To allow io_uring:

Option A: Pod-level seccomp (Kubernetes 1.19+)

spec:
  securityContext:
    seccompProfile:
      type: Unconfined    # or use a custom profile

Option B: Custom seccomp profile

Place the profile on each node at /var/lib/kubelet/seccomp/ferricstore.json:

{
  "defaultAction": "SCMP_ACT_ALLOW",
  "syscalls": [
    {"names": ["io_uring_setup", "io_uring_enter", "io_uring_register"],
     "action": "SCMP_ACT_ALLOW"}
  ]
}

spec:
  securityContext:
    seccompProfile:
      type: Localhost
      localhostProfile: ferricstore.json

NVMe Storage

Use a StorageClass backed by local NVMe SSDs:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nvme-local
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nvme-pv
spec:
  capacity:
    storage: 100Gi
  storageClassName: nvme-local
  local:
    path: /mnt/nvme
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values: ["node-with-nvme"]

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ferricstore-data
spec:
  storageClassName: nvme-local
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 100Gi

Then in the deployment:

volumeMounts:
  - name: data
    mountPath: /data
volumes:
  - name: data
    persistentVolumeClaim:
      claimName: ferricstore-data

CPU Pinning

For consistent latency, pin FerricStore pods to dedicated CPUs:

resources:
  requests:
    cpu: "4"
    memory: "8Gi"
  limits:
    cpu: "4"
    memory: "8Gi"

With static CPU manager policy on the kubelet, this guarantees exclusive cores.

Performance Checklist

Before benchmarking or going to production:

• [ ] io_uring enabled — check with cat /proc/sys/kernel/io_uring_disabled (should be 0)
• [ ] NVMe direct mount — not Docker overlay or network block storage
• [ ] CPU pinning — FerricStore on dedicated cores, not shared
• [ ] No swap — vm.swappiness=0 or mem_swappiness: 0
• [ ] Network — private network between cluster nodes, 10Gbps+
• [ ] Shard count — matches CPU count (default behavior)
• [ ] Protected mode — disabled if behind a firewall, or configure ACL