A lightweight, embedded, LSM-tree database for Elixir.
Goblin is a persistent key-value store with ACID transactions, crash recovery, and automatic background compaction. It runs inside your application's supervision tree.
Starting a database
{:ok, db} = Goblin.start_link(
name: MyApp.DB,
data_dir: "/path/to/db"
)Basic operations
Goblin.put(db, :alice, "Alice")
Goblin.get(db, :alice)
# => "Alice"
Goblin.remove(db, :alice)
Goblin.get(db, :alice)
# => nilBatch operations
Goblin.put_multi(db, [{:alice, "Alice"}, {:bob, "Bob"}])
Goblin.get_multi(db, [:alice, :bob])
# => [{:alice, "Alice"}, {:bob, "Bob"}]Transactions
Goblin.transaction(db, fn tx ->
counter = Goblin.Tx.get(tx, :counter, default: 0)
tx
|> Goblin.Tx.put(:counter, counter + 1)
|> Goblin.Tx.commit()
end)
# => :okSee start_link/1 for configuration options.
Summary
Functions
Returns whether background compaction is currently running.
Exports a snapshot of the database as a .tar.gz archive.
Returns whether a memory-to-disk flush is currently running.
Retrieves the value associated with a key.
Retrieves values for multiple keys in a single read.
Writes a key-value pair to the database.
Writes multiple key-value pairs in a single transaction.
Performs a read-only transaction.
Removes a key from the database.
Removes multiple keys from the database in a single transaction.
Returns a stream of key-value pairs, optionally bounded by a range. Captures snapshots at enumeration.
Starts the database, see start_link/1 for more details.
Starts the database.
Stops the database.
Executes a read-write transaction.
Functions
@spec child_spec(keyword()) :: Supervisor.child_spec()
@spec compacting?( :gen_statem.server_ref(), keyword() ) :: boolean()
Returns whether background compaction is currently running.
@spec export(:gen_statem.server_ref(), Path.t(), keyword()) :: {:ok, Path.t()} | {:error, term()}
Exports a snapshot of the database as a .tar.gz archive.
The archive can be unpacked and used as the data_dir for a new
database instance, acting as a backup.
The export is run inside the server, thus blocking file deletion and writes until completed.
Parameters
db- The database server (PID or registered name)export_dir- Directory to place the exported.tar.gzfile
Returns
{:ok, export_path}- Path to the created archive{:error, reason}- If an error occurred
Examples
Goblin.export(db, "/backups")
# => {:ok, "/backups/goblin_20260220T120000Z.tar.gz"}@spec flushing?( :gen_statem.server_ref(), keyword() ) :: boolean()
Returns whether a memory-to-disk flush is currently running.
Retrieves the value associated with a key.
Returns the default value if the key is not found.
Parameters
db- The database server (PID or registered name)key- The key to look upopts- A keyword list with the following options (default:[])::tag- Tag the key is namespaced under:default- Value to return ifkeyis not found (default:nil)
Returns
- The value associated with the key, or
defaultif not found
Examples
Goblin.get(db, :alice)
# => "Alice"
Goblin.get(db, :nonexistent)
# => nil
Goblin.get(db, :nonexistent, default: :not_found)
# => :not_found
Goblin.get(db, :alice, tag: :admins)
# => "Alice"Retrieves values for multiple keys in a single read.
Keys not found in the database are excluded from the result.
Parameters
db- The database server (PID or registered name)keys- A list of keys to look upopts- A keyword list with the following options (default:[])::tag- Tag the keys are namespaced under
Returns
- A list of
{key, value}tuples for keys found, in unspecified order
Examples
Goblin.get_multi(db, [:alice, :bob])
# => [{:alice, "Alice"}, {:bob, "Bob"}]
Goblin.get_multi(db, [:alice, :nonexistent])
# => [{:alice, "Alice"}]Writes a key-value pair to the database.
Parameters
db- The database server (PID or registered name)key- Any Elixir term to use as the keyvalue- Any Elixir term to be associated withkeyopts- A keyword list with the following options (default:[])::tag- Tag to namespace the key under:timeout- Timeout (in milliseconds) for the calls (default:5000)
Returns
:ok
Examples
Goblin.put(db, :alice, "Alice")
# => :ok
Goblin.put(db, :alice, "Alice", tag: :admins)
# => :okWrites multiple key-value pairs in a single transaction.
Parameters
db- The database server (PID or registered name)pairs- A list of{key, value}tuplesopts- A keyword list with the following options (default:[])::tag- Tag to namespace the keys under:timeout- Timeout (in milliseconds) for the calls (default:5000)
Returns
:ok
Examples
Goblin.put_multi(db, [{:alice, "Alice"}, {:bob, "Bob"}, {:charlie, "Charlie"}])
# => :okPerforms a read-only transaction.
A snapshot is taken to provide a consistent mvcc of the database. Multiple readers run concurrently without blocking each other. Attempting to write within a read transaction raises.
Parameters
db- The database server (PID or registered name)callback- A function that takes aGoblin.Tx.t()struct
Returns
- The return value of
callback
Examples
Goblin.read(db, fn tx ->
alice = Goblin.Tx.get(tx, :alice)
bob = Goblin.Tx.get(tx, :bob)
{alice, bob}
end)
# => {"Alice", "Bob"}Removes a key from the database.
Parameters
db- The database server (PID or registered name)key- The key to removeopts- A keyword list with the following options (default:[])::tag- Tag the key is namespaced under:timeout- Timeout (in milliseconds) for the calls (default:5000)
Returns
:ok
Examples
Goblin.remove(db, :alice)
# => :ok
Goblin.get(db, :alice)
# => nilRemoves multiple keys from the database in a single transaction.
Parameters
db- The database server (PID or registered name)keys- A list of keys to removeopts- A keyword list with the following options (default:[])::tag- Tag the keys are namespaced under:timeout- Timeout (in milliseconds) for the calls (default:5000)
Returns
:ok
Examples
Goblin.remove_multi(db, [:alice, :bob, :charlie])
# => :okReturns a stream of key-value pairs, optionally bounded by a range. Captures snapshots at enumeration.
Entries are sorted by key in ascending order.
Both min and max are inclusive.
Parameters
db- The database server (PID or registered name)opts- Keyword list of options::min- Minimum key, inclusive (optional):max- Maximum key, inclusive (optional):tag- Tag to filter by (optional)
Returns
- A stream of
{key, value}tuples
Examples
Goblin.scan(db) |> Enum.to_list()
# => [{:alice, "Alice"}, {:bob, "Bob"}, {:charlie, "Charlie"}]
Goblin.scan(db, min: :bob) |> Enum.to_list()
# => [{:bob, "Bob"}, {:charlie, "Charlie"}]
Goblin.scan(db, min: :alice, max: :bob) |> Enum.to_list()
# => [{:alice, "Alice"}, {:bob, "Bob"}]
scan = Goblin.scan(db)
Enum.to_list(scan)
# => []
Goblin.put(db, :alice, "Alice")
Enum.to_list(scan)
# => [{:alice, "Alice"}]Starts the database, see start_link/1 for more details.
Starts the database.
Creates the data_dir if it does not exist.
Options
:name- Registered name for the database (optional, defaults toGoblin):data_dir- Directory path for database files (required):mem_limit- Bytes to buffer in memory before flushing to disk (default: 64 MB):bf_fpp- Bloom filter false positive probability (default: 0.01)
Returns
{:ok, pid}- On successful start{:error, reason}- On failure
Examples
{:ok, db} = Goblin.start_link(
name: MyApp.DB,
data_dir: "/var/lib/myapp/db"
)@spec stop(:gen_statem.server_ref(), term(), timeout()) :: :ok
Stops the database.
Executes a read-write transaction.
Transactions are executed serially and are ACID-compliant.
The provided function receives a transaction struct and must return
{:commit, tx, reply} to commit, or {:abort, reply} to abort.
Parameters
db- The database server (PID or registered name)callback- A function that takes aGoblin.Tx.t()and returns a transaction resultopts- A keyword list with options::timeout- Timeout (in milliseconds) for the calls (default:5000)
Returns
reply- The reply from{:commit, tx, reply}when committed{:error, :aborted}- When the transaction is aborted
Examples
Goblin.transaction(db, fn tx ->
counter = Goblin.Tx.get(tx, :counter, default: 0)
tx
|> Goblin.Tx.put(:counter, counter + 1)
|> Goblin.Tx.commit()
end)
# => :ok
Goblin.transaction(db, fn tx ->
tx
|> Goblin.Tx.abort()
end)
# => :error