cubdb v0.1.4 CubDB View Source

CubDB is a pure-Elixir embedded key-value database, designed for simplicity. It runs locally, and is backed by a single file.

Both keys and values can be any Elixir (or Erlang) term.

The CubDB database file uses an immutable data structure that ensures robustness to data corruption: entries are never changed in-place, and writes are atomic.

Read operations are performed on immutable "snapshots", so they are always consistent, run concurrently, and do not block write operations, nor are blocked by them.

CubDB is a pure-Elixir embedded key-value database, designed for simplicity. It runs locally, and is backed by a single file.

Both keys and values can be any Elixir (or Erlang) term, so no serialization and de-serialization is necessary.

The CubDB database file uses an immutable data structure that ensures robustness to data corruption: entries are never changed in-place, and writes are atomic.

Read operations are performed on immutable "snapshots", so they are always consistent, run concurrently, and do not block write operations, nor are blocked by them.

Usage

Start CubDB by specifying a directory for its database file (if not existing, it will be created):

{:ok, db} = CubDB.start_link("my/data/directory")

The get/2, put/3, and delete/2 functions work as you probably expect:

CubDB.put(db, :foo, "some value")
#=> :ok

CubDB.get(db, :foo)
#=> "some value"

CubDB.delete(db, :foo)
#=> :ok

CubDB.get(db, :foo)
#=> nil

Range of keys are retrieved using select/3:

for {key, value} <- [a: 1, b: 2, c: 3, d: 4, e: 5, f: 6, g: 7, h: 8] do
  CubDB.put(db, key, value)
end

CubDB.select(db, min_key: :b, max_key: :e)
#=> {:ok, [b: 2, c: 3, d: 4, e: 5]}

But select/3 can do much more than that. It can apply a pipeline of operations (map, filter, take, drop and more) to the selected entries, it can select the entries in normal or reverse order, and it can reduce the result using an arbitrary function:

# Take the sum of the last 3 even values:
CubDB.select(db,
  reverse: true,
  pipe: [
    map: fn {_key, value} -> value end,
    filter: &Integer.is_even/1,
    take: 3
  ],
  reduce: fn n, sum -> sum + n end
)
#=> {:ok, 18}

As CubDB uses an immutable data structure, the data file will grow with each write operation. Occasionally, it is adviseable to run a compaction to optimize the file size and re-claim disk space. Compaction is started manually by calling compact/1, and runs in the background, without blocking other operations:

CubDB.compact(db)
#=> :ok

Link to this section Summary

Functions

Returns a specification to start this module under a supervisor.

Runs a database compaction.

Deletes the entry associated to key from the database.

Get the value associated to key from the database.

Returns a tuple indicating if key is associated to a value in the database.

Writes an entry in the database, associating key to value.

Selects a range of entries from the database, and optionally performs a pipeline of operations on them.

Returns the number of entries present in the database.

Starts the CubDB database without a link.

Starts the CubDB database process linked to the current process.

Link to this section Functions

Returns a specification to start this module under a supervisor.

See Supervisor.

Link to this function

compact(db) View Source
compact(GenServer.server()) :: :ok | {:error, binary()}

Runs a database compaction.

As write operations are performed on a database, its file grows. Occasionally, a compaction operation can be ran to shrink the file to its optimal size. Compaction is ran in the background and does not block operations.

Only one compaction operation can run at any time, therefore if this function is called when a compaction is already running, it returns {:error, :pending_compaction}.

Link to this function

delete(db, key) View Source
delete(GenServer.server(), any()) :: :ok

Deletes the entry associated to key from the database.

If key was not present in the database, nothing is done.

Get the value associated to key from the database.

If no value is associated with key, nil is returned.

Link to this function

has_key?(db, key) View Source
has_key?(GenServer.server(), any()) :: {boolean(), any()}

Returns a tuple indicating if key is associated to a value in the database.

If key is associated to a value, it returns {true, value}. Otherwise, it returns {false, nil}

Link to this function

put(db, key, value) View Source
put(GenServer.server(), any(), any()) :: :ok

Writes an entry in the database, associating key to value.

If key was already present, it is overwritten.

Link to this function

select(db, options \\ [], timeout \\ 5000) View Source
select(GenServer.server(), Keyword.t(), timeout()) ::
  {:ok, any()} | {:error, Exception.t()}

Selects a range of entries from the database, and optionally performs a pipeline of operations on them.

It returns {:ok, result} if successful, or {:error, exception} if an exception is raised.

Options

The min_key and max_key the range of entriethat is selected. All entries that have a key greater or equal than min_key and less or equal then max_key are selected. One or both of min_key and max_key can be omitted or set to nil, in which case the range is open-ended.

The reverse option, when set to true, causes the entries to be selected and traversed in reverse order.

The pipe option specifies an optional list of operations performed sequentially on the selected entries. The given order of operations is respected. The available operations, specified as tuples, are:

  • {:filter, fun} filters entries for which fun returns a truthy value

  • {:map, fun} maps each entry to the value returned by the function fun

  • {:take, n} takes the first n entries

  • {:drop, n} skips the first n entries

  • {:take_while, fun} takes entries while fun returns a truthy value

  • {:drop_while, fun} skips entries while fun returns a truthy value

The reduce option specifies how the selected entries are aggregated. If reduce is omitted, the entries are returned as a list. If reduce is a function, it is used to reduce the collection of entries. If reduce is a tuple, the first element is the starting value of the reduction, and the second is the reducing function.

Examples

To select all entries with keys between :a and :c as a list of {key, value} we can do:

{:ok, entries} = CubDB.select(db, min_key: :a, max_key: :c)

To select the last 3 entries, we can do:

{:ok, entries} = CubDB.select(db, reverse: true, pipe: [take: 3])

If we want to obtain the sum of the first 10 positive numeric values associated to keys from :a to :f, we can do:

{:ok, sum} = CubDB.select(db,
  min_key: :a,
  max_key: :f,
  pipe: [
    map: fn {_key, value} -> value end, # map values
    filter: fn n -> is_number(n) and n > 0 end # only positive numbers
    take: 10, # take only the first 10 entries in the range
  ],
  reduce: fn n, sum -> sum + n end # reduce to the sum of selected values
)

Returns the number of entries present in the database.

Starts the CubDB database without a link.

See start_link/2 for more informations.

Link to this function

start_link(data_dir, options \\ []) View Source

Starts the CubDB database process linked to the current process.

The data_dir argument is the directory path where the database files will be stored. If it does not exist, it will be created. Only one CubDB instance can run per directory, so if you run several databases, they should each use their own separate data directory.

The options are passed to GenServer.start_link/3.