Concord.Index (Concord v3.0.2)

Copy Markdown View Source

Secondary index support for efficient value-based queries.

Secondary indexes enable fast lookups by specific value fields without scanning all keys. Each index maintains a mapping from indexed values to the keys that contain those values.

Features

  • Automatic Maintenance: Indexes update automatically on put/delete
  • Multiple Indexes: Support for multiple indexes per store
  • Declarative Extractors: Define indexes with data specs (safe for replication)
  • Efficient Lookups: O(1) lookup by indexed value
  • Multi-value Support: Index multiple values per key (e.g., tags)
# Index on a map key
:ok = Concord.Index.create("users_by_email", {:map_get, :email})

# Index on a nested path
:ok = Concord.Index.create("by_city", {:nested, [:address, :city]})

# Index on the raw value
:ok = Concord.Index.create("by_value", {:identity})

Anonymous function extractors are rejected because executing code references from a replicated log is not deterministic across nodes or code versions.

Summary

Types

extractor()

Declarative extractor specification

index_name()

Index name (unique identifier)

index_value()

Value to index (must be comparable)

Functions

create(name, extractor, opts \\ [])

Creates a new secondary index.

drop(name, opts \\ [])

Drops an existing secondary index.

list(opts \\ [])

Lists all secondary indexes.

lookup(name, value, opts \\ [])

Looks up keys by indexed value.

reindex(name, opts \\ [])

Rebuilds an index from all existing keys. Uses the declarative extractor spec stored in the state machine.

Types

extractor()

@type extractor() :: Concord.Index.Extractor.spec()

Declarative extractor specification

index_name()

@type index_name() :: String.t()

Index name (unique identifier)

index_value()

@type index_value() :: term()

Value to index (must be comparable)

Functions

create(name, extractor, opts \\ [])

@spec create(index_name(), extractor(), keyword()) :: :ok | {:error, term()}

Creates a new secondary index.

Accepts a declarative extractor spec. Anonymous functions return {:error, :invalid_extractor} before a storage command is issued.

:ok = Concord.Index.create("by_email", {:map_get, :email})
:ok = Concord.Index.create("by_city", {:nested, [:address, :city]})
:ok = Concord.Index.create("by_value", {:identity})
:ok = Concord.Index.create("by_first", {:element, 0})

Options

  • :reindex - If true, reindex all existing keys (default: false)
  • :timeout - Operation timeout in milliseconds (default: 5000)

drop(name, opts \\ [])

@spec drop(
  index_name(),
  keyword()
) :: :ok | {:error, term()}

Drops an existing secondary index.

list(opts \\ [])

@spec list(keyword()) :: {:ok, [String.t()]} | {:error, term()}

Lists all secondary indexes.

lookup(name, value, opts \\ [])

@spec lookup(index_name(), index_value(), keyword()) ::
  {:ok, [String.t()]} | {:error, term()}

Looks up keys by indexed value.

reindex(name, opts \\ [])

@spec reindex(
  index_name(),
  keyword()
) :: :ok | {:error, term()}

Rebuilds an index from all existing keys. Uses the declarative extractor spec stored in the state machine.