Elixir API Guide

Complete guide to using Concord's Elixir API for read consistency, conditional updates, query language, and value compression.

Read Consistency Aliases

Concord accepts :eventual, :leader, and :strong as consistency names for API compatibility. The VSR engine currently routes every name through the same quorum-confirmed read barrier. All three are linearizable; they do not select different replicas, freshness guarantees, or performance profiles.

Configuration

Set the default accepted name in config/config.exs:

config :concord,
  default_read_consistency: :leader  # :eventual, :leader, or :strong

The setting preserves caller compatibility but does not change the current VSR read path.

Per-Operation Alias

Concord.get("key", consistency: :eventual)
Concord.get_many(["k1", "k2"], consistency: :strong)
Concord.get_with_ttl("key", consistency: :leader)
Concord.ttl("key", consistency: :eventual)
Concord.get_all(consistency: :strong)
Concord.get_all_with_ttl(consistency: :eventual)
Concord.status(consistency: :leader)

Telemetry Integration

Read telemetry includes the requested compatibility name:

:telemetry.attach(
  "my-handler",
  [:concord, :api, :get],
  fn _event, %{duration: duration}, %{consistency: consistency}, _config ->
    Logger.info("Read requested #{consistency}; duration=#{duration}ns")
  end,
  nil
)

The :consistency metadata value reports what the caller requested. It does not identify a distinct VSR execution path or guarantee.

Query Module Consistency

Concord.Query functions (keys/1, where/1, count/1, delete_where/1) do not currently accept a :consistency option. They delegate to Concord.get_all/1 and Concord.get_many/2 using the globally configured compatibility name:

config :concord,
  default_read_consistency: :leader  # :eventual, :leader, or :strong

Changing this name does not change the VSR read path or guarantee.

Index Lookups

Concord.Index.lookup/3 uses the replicated VSR query barrier. This ensures index lookups return results consistent with the latest writes.

Secondary Indexes

Secondary indexes enable efficient value-based lookups without scanning all keys. Concord maintains indexes automatically as values are inserted, updated, and deleted.

Extractor Specs

Indexes are defined using declarative extractor specs -- plain tuples that describe how to extract the indexed value from stored data. There are four supported spec types:

{:map_get, key} -- Flat map field

Extracts a single top-level key from a map value using Map.get/2:

# Given stored values like %{email: "alice@example.com", name: "Alice"}
Concord.Index.create("users_by_email", {:map_get, :email})

# After inserting data:
Concord.put("user:1", %{email: "alice@example.com", name: "Alice"})
Concord.put("user:2", %{email: "bob@example.com", name: "Bob"})

# Look up by indexed value:
{:ok, ["user:1"]} = Concord.Index.lookup("users_by_email", "alice@example.com")

{:nested, path} -- Nested map path

Extracts a value from a nested map structure using get_in/2:

# Given stored values like %{address: %{city: "Portland", state: "OR"}}
Concord.Index.create("users_by_city", {:nested, [:address, :city]})

Concord.put("user:1", %{name: "Alice", address: %{city: "Portland", state: "OR"}})
Concord.put("user:2", %{name: "Bob", address: %{city: "Seattle", state: "WA"}})

{:ok, ["user:1"]} = Concord.Index.lookup("users_by_city", "Portland")

{:identity} -- Raw value

Indexes the entire stored value as-is. Useful when values are simple scalars (strings, integers):

Concord.Index.create("by_status", {:identity})

Concord.put("order:1", "pending")
Concord.put("order:2", "shipped")
Concord.put("order:3", "pending")

{:ok, ["order:1", "order:3"]} = Concord.Index.lookup("by_status", "pending")

{:element, n} -- Tuple element

Extracts the nth element (zero-indexed) from a tuple value:

# Given stored values like {"electronics", "laptop", 999}
Concord.Index.create("by_category", {:element, 0})

Concord.put("product:1", {"electronics", "laptop", 999})
Concord.put("product:2", {"clothing", "shirt", 29})
Concord.put("product:3", {"electronics", "phone", 799})

{:ok, ["product:1", "product:3"]} = Concord.Index.lookup("by_category", "electronics")

Managing Indexes

# Create an index
:ok = Concord.Index.create("users_by_email", {:map_get, :email})

# Create an index and rebuild it from all existing data
:ok = Concord.Index.create("users_by_email", {:map_get, :email}, reindex: true)

# List all indexes
{:ok, index_names} = Concord.Index.list()

# Look up keys by indexed value
{:ok, keys} = Concord.Index.lookup("users_by_email", "alice@example.com")

# Rebuild an index from scratch
:ok = Concord.Index.reindex("users_by_email")

# Drop an index
:ok = Concord.Index.drop("users_by_email")

Multi-Value Indexing

When the extractor returns a list, each element is indexed separately. This is useful for tagging:

Concord.Index.create("by_tag", {:map_get, :tags})

Concord.put("post:1", %{title: "Elixir Tips", tags: ["elixir", "programming"]})
Concord.put("post:2", %{title: "VSR Consensus", tags: ["distributed", "elixir"]})

{:ok, ["post:1", "post:2"]} = Concord.Index.lookup("by_tag", "elixir")
{:ok, ["post:2"]} = Concord.Index.lookup("by_tag", "distributed")

Warning: No Anonymous Functions

Do not use anonymous functions as extractors. They cause :badfun errors on deserialization across code versions. Always use declarative tuple specs.

# BAD -- will break after code upgrades or on other cluster nodes
Concord.Index.create("by_email", fn user -> user.email end)

# GOOD -- safe for replication and snapshots
Concord.Index.create("by_email", {:map_get, :email})

Anonymous functions are serialized into the replicated log and snapshots. When a node loads a snapshot produced by a different code version, the function reference can be invalid and raise :badfun. Declarative specs are plain data (tuples of atoms, binaries, and integers) and are safe to deserialize.

Conditional Updates (Compare-and-Swap)

Atomic conditional operations for CAS, distributed locks, and optimistic concurrency control.

Compare-and-Swap with Expected Value

# Initialize counter
:ok = Concord.put("counter", 0)

# Read current value
{:ok, current} = Concord.get("counter")

# Update only if value hasn't changed
case Concord.put_if("counter", current + 1, expected: current) do
  :ok -> IO.puts("Counter updated to #{current + 1}")
  {:error, :condition_failed} -> IO.puts("Conflict, retrying...")
  {:error, :not_found} -> IO.puts("Key no longer exists")
end

# Conditional delete
:ok = Concord.put("session", "user-123")
:ok = Concord.delete_if("session", expected: "user-123")

Predicate-Based Conditions

# Version-based updates (optimistic locking)
:ok = Concord.put("config", %{version: 1, settings: %{enabled: true}})

new_config = %{version: 2, settings: %{enabled: false}}
:ok = Concord.put_if("config", new_config,
  condition: fn current -> current.version < new_config.version end
)

# Conditional delete based on age
cutoff = ~U[2025-01-01 00:00:00Z]
:ok = Concord.delete_if("temp_file",
  condition: fn file -> DateTime.compare(file.created_at, cutoff) == :lt end
)

Distributed Lock Pattern

defmodule DistributedLock do
  @lock_key "my_critical_resource"
  @lock_ttl 30

  def acquire(owner_id) do
    case Concord.get(@lock_key) do
      {:error, :not_found} ->
        Concord.put(@lock_key, owner_id, ttl: @lock_ttl)
        {:ok, :acquired}
      {:ok, ^owner_id} ->
        {:ok, :already_owned}
      {:ok, _other} ->
        {:error, :locked}
    end
  end

  def release(owner_id) do
    case Concord.delete_if(@lock_key, expected: owner_id) do
      :ok -> {:ok, :released}
      {:error, :condition_failed} -> {:error, :not_owner}
      {:error, :not_found} -> {:error, :not_locked}
    end
  end

  def with_lock(owner_id, fun) do
    case acquire(owner_id) do
      {:ok, _} ->
        try do
          fun.()
        after
          release(owner_id)
        end
      {:error, reason} ->
        {:error, reason}
    end
  end
end

Optimistic Concurrency Control

defmodule BankAccount do
  def transfer(from_account, to_account, amount) do
    {:ok, from_balance} = Concord.get(from_account)
    {:ok, to_balance} = Concord.get(to_account)

    if from_balance >= amount do
      with :ok <- Concord.put_if(from_account, from_balance - amount, expected: from_balance),
           :ok <- Concord.put_if(to_account, to_balance + amount, expected: to_balance) do
        {:ok, :transferred}
      else
        {:error, :condition_failed} ->
          transfer(from_account, to_account, amount)  # Retry
        error -> error
      end
    else
      {:error, :insufficient_funds}
    end
  end
end

API Options

Condition options (required, mutually exclusive):

• :expected — Exact value match (== comparison)
• :condition — Predicate function receiving current value

Additional options (for put_if/3):

• :ttl — TTL in seconds on success
• :timeout — Operation timeout in ms (default: 5000)

Return values:

• :ok — Condition met, operation succeeded
• {:error, :condition_failed} — Value doesn't match
• {:error, :not_found} — Key doesn't exist or expired
• {:error, :missing_condition} — No condition provided
• {:error, :conflicting_conditions} — Both :expected and :condition provided

TTL Interaction

Conditional operations treat expired keys as not found:

:ok = Concord.put("temp", "value", ttl: 1)
Process.sleep(2000)
{:error, :not_found} = Concord.put_if("temp", "new", expected: "value")

Query Language

Pattern matching, range queries, and filtering for efficient data retrieval.

Key Matching

# Prefix matching
{:ok, keys} = Concord.Query.keys(prefix: "user:")

# Suffix matching
{:ok, keys} = Concord.Query.keys(suffix: ":admin")

# Contains substring
{:ok, keys} = Concord.Query.keys(contains: "2024-02")

# Regex pattern
{:ok, keys} = Concord.Query.keys(pattern: ~r/user:\d{3}/)

Range Queries

# Lexicographic range (inclusive)
{:ok, keys} = Concord.Query.keys(range: {"user:100", "user:200"})

# Date range queries
{:ok, keys} = Concord.Query.keys(range: {"order:2024-01-01", "order:2024-12-31"})

Value Filtering

{:ok, pairs} = Concord.Query.where(
  prefix: "product:",
  filter: fn {_k, v} -> v.price > 100 end
)

{:ok, pairs} = Concord.Query.where(
  prefix: "user:",
  filter: fn {_k, v} -> v.age >= 30 and v.role == "admin" end
)

Pagination

{:ok, keys} = Concord.Query.keys(prefix: "user:", limit: 50)
{:ok, keys} = Concord.Query.keys(prefix: "user:", offset: 100, limit: 50)

Count and Delete

{:ok, count} = Concord.Query.count(prefix: "temp:")
{:ok, deleted_count} = Concord.Query.delete_where(prefix: "temp:")
{:ok, count} = Concord.Query.delete_where(range: {"old:2020-01-01", "old:2020-12-31"})

Combined Filters

{:ok, keys} = Concord.Query.keys(
  prefix: "user:",
  pattern: ~r/\d{3}/,
  limit: 10
)

Value Compression

Automatic compression for large values to reduce memory usage.

Configuration

Compression is enabled by default:

config :concord,
  compression: [
    enabled: true,
    algorithm: :zlib,        # :zlib (faster) or :gzip (better ratio)
    threshold_bytes: 1024,   # Compress values > 1KB
    level: 6                 # 0-9 (0=none, 9=max)
  ]

Transparent Operation

# Large value — automatically compressed on put
large_data = String.duplicate("x", 10_000)
Concord.put("large_key", large_data)

# Automatically decompressed on get
{:ok, value} = Concord.get("large_key")
# Returns original uncompressed value

Per-Operation Override

# Force compression regardless of size
Concord.put("small_key", "small value", compress: true)

# Disable compression for this operation
Concord.put("large_key", large_value, compress: false)

Compression Statistics

stats = Concord.Compression.stats(large_data)
# %{
#   original_size: 10_047,
#   compressed_size: 67,
#   compression_ratio: 0.67,
#   savings_bytes: 9_980,
#   savings_percent: 99.33
# }

Performance

Trade-offs: ~5-15% CPU overhead, 60-98% memory reduction, ~0.1-1ms added latency.

API Reference

Core Operations

Concord.put(key, value, opts \\ [])
# Options: :timeout, :token, :ttl, :compress

Concord.get(key, opts \\ [])
# Returns: {:ok, value} | {:error, :not_found} | {:error, reason}
# Options: :timeout, :token, :consistency

Concord.delete(key, opts \\ [])
# Returns: :ok | {:error, reason}

Concord.get_all(opts \\ [])
# Returns: {:ok, map}

Concord.status(opts \\ [])
# Returns: {:ok, %{cluster: ..., storage: ..., node: ...}}

Concord.members()
# Returns: {:ok, [member_ids]}

Batch Operations

Concord.put_many([{key, value} | {key, value, ttl}], opts)
Concord.get_many([keys], opts)
Concord.delete_many([keys], opts)
Concord.touch_many([{key, ttl_seconds}], opts)

Max batch size: 500 items.

TTL Operations

Concord.put(key, value, ttl: seconds)
Concord.touch(key, additional_ttl_seconds, opts)
Concord.ttl(key, opts)
Concord.get_with_ttl(key, opts)
Concord.get_all_with_ttl(opts)

Conditional Operations

Concord.put_if(key, value, expected: current_value)
Concord.put_if(key, value, condition: fn current -> ... end)
Concord.delete_if(key, expected: current_value)
Concord.delete_if(key, condition: fn current -> ... end)

Common Options

• :timeout — Operation timeout in ms (default: 5000)
• :token — Authentication token (required when auth enabled)
• :consistency — Read consistency (:eventual, :leader, :strong)
• :ttl — Time-to-live in seconds
• :compress — Override auto-compression (true/false)

Error Types

:timeout              # Operation timed out
:unauthorized         # Invalid or missing auth token
:cluster_not_ready    # Cluster not initialized
:invalid_key          # Key validation failed
:not_found            # Key doesn't exist
:cluster_not_ready    # VSR cluster is not ready
:condition_failed     # Conditional update failed