TypeDB (TypeDB v0.10.0)

Copy Markdown View Source

A driver for TypeDB 3.12 and newer, built on the TypeDB HTTP API.

Older 3.x is not supported; see the README for what breaks and why.

Getting started

Add the connection to your supervision tree:

children = [
  {TypeDB,
   url: "http://localhost:8000",
   username: "admin",
   password: System.fetch_env!("TYPEDB_PASSWORD")}
]

Supervisor.start_link(children, strategy: :one_for_one)

Then query. TypeDB is the default connection name:

TypeDB.query!(TypeDB, "social", """
  match
    $p isa person, has name $name;
  select $name;
""")
|> Enum.map(&TypeDB.ConceptRow.typed_value(&1, "name"))
#=> ["Alice", "Bob"]

The two ways to run a query

query/4 runs a single query in its own transaction — TypeDB opens it, runs the query, and commits or closes it in one round trip. Reads never commit; writes and schema changes commit by default, which commit: false turns off. Every function takes the connection's registered name as its first argument; conn below is that name, TypeDB unless you chose another.

TypeDB.query(conn, "social", "insert $p isa person, has name 'Alice';")

transaction/5 opens a transaction you can run several queries in, committing once at the end:

TypeDB.transaction(conn, "social", :write, fn tx ->
  TypeDB.Transaction.query!(tx, "insert $p isa person, has name 'Alice';")
  TypeDB.Transaction.query!(tx, "insert $p isa person, has name 'Bob';")
end)

The block commits on success and rolls back on error or exception. A :read block never commits.

Answers

Every query returns a TypeDB.Answer — see that module for the three shapes and how to consume them.

Errors

Every function in this module and in TypeDB.Database, TypeDB.User, TypeDB.Server and TypeDB.Transaction that can fail has both a {:ok, _} | {:error, %TypeDB.Error{}} form and a ! form that raises. TypeDB.Error carries TypeDB's own error :code, which is what you want to branch on.

The one exception is transaction/5: it returns whatever your block returned, so its {:error, _} may be your own value rather than an exception, and a bang form would have to guess whether to raise it.

How long a call can take

Four options interact, and only the last of them bounds the call rather than one attempt. With the defaults:

defaultwhat it bounds
:connect_timeout10_000opening the socket
:timeout60_000waiting for one response
:max_retries1how many extra attempts
:retry_max_delay5_000one wait between attempts
:deadline:infinitythe whole call

So a retryable call costs at worst

(max_retries + 1) * (connect_timeout + timeout) + max_retries * retry_max_delay

which is a little over two minutes by default, plus one more connect_timeout + timeout if the call has to mint a token first. Raising :max_retries multiplies the first term — that is the arithmetic :deadline exists for, since setting it makes the whole expression irrelevant: no call outlives its budget, each attempt is given only what the budget has left, and a retry that could not finish inside it is not started.

Retries and their backoffs happen in the calling process. Nothing is queued behind a connection, but nothing is running in the caller either.

Logging

The driver logs sparingly, and nothing at all on the happy path — use TypeDB.Telemetry for that. This is everything it can say:

levelwhen
debuga request is about to be retried
debuga connection process received a message it does not understand
warningretries were exhausted and the call is giving up
warninga token renewal failed
errora connection's transport died and the connection is stopping

TypeDB.HTTP.Httpc additionally warns once at start-up when it can find no OS trust store, carrying :typedb_adapter rather than :typedb_connection: it happens while the adapter is being built, before there is a connection to name or a :log_level to consult.

Set :log_level on the connection to raise the floor, or :none to silence it — see TypeDB.Config. It applies per connection, so one noisy connection can be quietened without filtering the whole application's Logger by module.

Every line carries :typedb_connection in its Logger metadata, and the retry and give-up lines additionally carry :typedb_method, :typedb_path, :typedb_attempt and :typedb_error_kind. Configure your backend to keep them:

config :logger, :default_formatter,
  metadata: [:typedb_connection, :typedb_error_kind]

Credentials never appear in a log line or in metadata; see TypeDB.Config for how the connection keeps them out of crash reports too.

Concurrency

Requests run in the calling process, so N processes issue N concurrent requests; the connection process is consulted only to mint or renew the auth token. Sockets are pooled by the HTTP adapter — :max_sessions on TypeDB.HTTP.Httpc caps how many are opened per host.

What this driver covers

Everything in the TypeDB HTTP API v1: sign-in and token renewal, databases, users, servers, version and health, explicit transactions, one-shot queries, and query analysis. TypeDB's gRPC-only features — database import/export and streaming answers — are not available over HTTP and so are not here.

What that costs you in practice, and the four other things worth knowing before they surprise you in production, are in the Limitations section of the README: answers arrive whole, a connection points at one server, retries block the caller, and one connection means one HTTP pool.

Summary

Types

A connection: the registered name of a TypeDB.Connection process.

Functions

Creates a database, raising on failure. See TypeDB.Database.create!/2.

Creates a database unless it already exists, raising on failure. See TypeDB.Database.create_if_not_exists!/3.

Lists databases, raising on failure. See TypeDB.Database.list!/1.

Deletes a database and all of its data. See TypeDB.Database.delete/2.

Deletes a database, raising on failure. See TypeDB.Database.delete!/2.

Returns :ok when the server is reachable. See TypeDB.Server.health/1.

Returns :ok when the server is reachable, raising otherwise. See TypeDB.Server.health!/1.

Runs a single query in a transaction of its own.

Runs a single query, raising TypeDB.Error on failure.

Returns true when conn can serve a request.

Starts a connection. See TypeDB.Config for options.

Stops a connection, by registered name or pid.

Walks a read query one page at a time, as a lazy Stream of TypeDB.ConceptRow.

Runs fun inside a transaction, committing on success.

Returns the server distribution and version. See TypeDB.Server.version/1.

Returns the server distribution and version, raising on failure. See TypeDB.Server.version!/1.

Types

conn()

@type conn() :: TypeDB.Connection.t()

A connection: the registered name of a TypeDB.Connection process.

Functions

create_database(conn, name, opts \\ [])

@spec create_database(conn(), String.t(), keyword()) ::
  :ok | {:error, TypeDB.Error.t()}

Creates a database. See TypeDB.Database.create/2.

create_database!(conn, name, opts \\ [])

@spec create_database!(conn(), String.t(), keyword()) :: :ok

Creates a database, raising on failure. See TypeDB.Database.create!/2.

create_database_if_not_exists(conn, name, opts \\ [])

@spec create_database_if_not_exists(conn(), String.t(), keyword()) ::
  :ok | {:error, TypeDB.Error.t()}

Creates a database unless it already exists. See TypeDB.Database.create_if_not_exists/3.

This is what a migration wants at boot. Without it on the facade, an application that works through TypeDB alone reaches for databases/2 and a membership test — two round trips, and a list of every database on the server to answer a question about one.

create_database_if_not_exists!(conn, name, opts \\ [])

@spec create_database_if_not_exists!(conn(), String.t(), keyword()) :: :ok

Creates a database unless it already exists, raising on failure. See TypeDB.Database.create_if_not_exists!/3.

databases(conn, opts \\ [])

@spec databases(
  conn(),
  keyword()
) :: {:ok, [String.t()]} | {:error, TypeDB.Error.t()}

Lists databases. See TypeDB.Database.list/1.

databases!(conn, opts \\ [])

@spec databases!(
  conn(),
  keyword()
) :: [String.t()]

Lists databases, raising on failure. See TypeDB.Database.list!/1.

delete_database(conn, name, opts \\ [])

@spec delete_database(conn(), String.t(), keyword()) ::
  :ok | {:error, TypeDB.Error.t()}

Deletes a database and all of its data. See TypeDB.Database.delete/2.

delete_database!(conn, name, opts \\ [])

@spec delete_database!(conn(), String.t(), keyword()) :: :ok

Deletes a database, raising on failure. See TypeDB.Database.delete!/2.

health(conn, opts \\ [])

@spec health(
  conn(),
  keyword()
) :: :ok | {:error, TypeDB.Error.t()}

Returns :ok when the server is reachable. See TypeDB.Server.health/1.

health!(conn, opts \\ [])

@spec health!(
  conn(),
  keyword()
) :: :ok

Returns :ok when the server is reachable, raising otherwise. See TypeDB.Server.health!/1.

query(conn, database, query, opts \\ [])

@spec query(conn(), String.t(), String.t(), keyword()) ::
  {:ok, TypeDB.Answer.t()} | {:error, TypeDB.Error.t()}

Runs a single query in a transaction of its own.

TypeDB opens the transaction, runs the query, then commits or closes it — one HTTP round trip in total, which makes this the cheapest way to run a standalone query.

Options

  • :transaction_type:read, :write or :schema. Defaults to :schema, the only type that accepts every kind of query.

    A :schema transaction takes TypeDB's exclusive, database-wide schema lock for the duration of the call. One-shot queries left on the default therefore serialise against each other, and against anything else holding that lock. Pass transaction_type: :read for reads and :write for data changes — both are concurrent — and leave the default to define and undefine. Narrowing also has the server reject an accidental write.

  • :commit — whether to commit a write or schema query. Defaults to true. Read queries never commit.

  • :given_rows — input rows for the query's given stage; see TypeDB.Transaction.query/3 for how to parameterise a query safely.

  • plus all query and transaction options from TypeDB.Options, and :timeout and :deadline — the first bounds one attempt, the second the whole call including retries. See TypeDB.Config.

Raises

Unlike the rest of this module, query/4 has two failure paths. Anything the server rejects comes back as {:error, %TypeDB.Error{}}; anything rejected before the request is built raises, because there is no request to fail:

  • ArgumentError for an invalid :transaction_type — that is a literal in your source, not data.
  • ArgumentError for an option this function does not accept. A misspelled key would otherwise be dropped and its default applied, so commmit: false would commit and a misspelled :given_rows would run the query with no rows at all.
  • ArgumentError for an option value the option cannot take — answer_count_limit: 0, timeout: "5000" — checked against the same rules TypeDB.Config applies to the connection. nil means "unset" throughout and is always accepted.
  • TypeDB.Error with kind :encode for a :given_rows value the driver cannot turn into a TypeDB wire value, including a negative TypeDB.Duration.

Examples

TypeDB.query(conn, "social", "match $p isa person; select $p;",
  transaction_type: :read,
  answer_count_limit: 100
)

# Dry run: execute the write, then throw it away.
TypeDB.query(conn, "social", "insert $p isa person;", commit: false)

# Parameterised, and therefore safe against TypeQL injection.
TypeDB.query(conn, "social", """
  given $n: string;
  insert $p isa person, has name == $n;
""", given_rows: [%{"n" => user_supplied_name}])

query!(conn, database, query, opts \\ [])

@spec query!(conn(), String.t(), String.t(), keyword()) :: TypeDB.Answer.t()

Runs a single query, raising TypeDB.Error on failure.

running?(conn)

@spec running?(conn()) :: boolean()

Returns true when conn can serve a request.

For callers who cannot let a missing connection raise — a health endpoint, a module that maps driver failures onto its own error type. Note that Process.whereis/1 is not the same question: the name is registered before the connection finishes starting. See TypeDB.Connection.running?/1.

if TypeDB.running?(conn), do: TypeDB.query(conn, "social", query)

This is about the connection process on this node. health/2 is the question about the server, and it needs a running connection to ask it.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Starts a connection. See TypeDB.Config for options.

stop(conn, reason \\ :normal, timeout \\ :infinity)

@spec stop(conn() | pid(), term(), timeout()) :: :ok

Stops a connection, by registered name or pid.

stream(conn, database, query, opts \\ [])

@spec stream(conn(), String.t(), String.t(), keyword()) :: Enumerable.t()

Walks a read query one page at a time, as a lazy Stream of TypeDB.ConceptRow.

query/4 gives you what fits: the HTTP API caps a read at 10,000 answers by default, and TypeDB.Answer.truncated?/1 is how you find out it happened. This walks the whole answer instead, and it is the only way to read one larger than the cap over this transport.

conn
|> TypeDB.stream("social", """
     match $p isa person, has name $n;
     sort $n;
     select $n;
   """)
|> Stream.map(&TypeDB.ConceptRow.typed_value(&1, "n"))
|> Enum.each(&IO.puts/1)

Being a Stream, it composes with everything that takes one — Stream.filter/2, Enum.take/2, Flow, GenStage — and nothing is fetched until something demands it.

One transaction, one snapshot

The whole walk runs inside a single :read transaction, which is what makes the pages add up to one answer rather than to several. Measured against 3.12.1: a walk that collected 30 rows saw none of the 30 a concurrent transaction inserted while it was walking, and the database held 60 at the end.

The transaction is closed when the stream finishes, and also when a consumer stops early — Enum.take/2, a Stream.filter/2 that finds what it wanted, an exception in the middle. That is Stream.resource/3 doing its job.

The walk has a deadline, and it belongs to the transaction

One transaction is also one budget. transaction_timeout_millis is a lifetime counted from the moment the transaction opens, not an idle timer, and asking for the next page does not extend it — measured, a transaction opened with 5 000 ms and queried every two seconds was gone at 6 s. Left unset, the server's default is 300 000 ms, so an unconfigured walk has five minutes to finish, however fast its pages arrive.

Fast consumers never meet this. Measured against 3.12.1 over 3,000 rows at page_size: 200 — sixteen requests:

consumertransaction_timeout_millisoutcome
as fast as it can4 0003,000 rows in 279 ms
2 ms per row4 000raises TSV12 at 4 282 ms, mid-walk
2 ms per row30 0003,000 rows in 9 198 ms

So the thing that runs out is wall-clock time between opening and the last page, and what spends it is usually the consumer rather than the driver. If a walk might take longer than five minutes — a slow consumer, a costly sort, or simply a great many rows — pass transaction_timeout_millis to say how long, and pass it generously: an over-long budget costs what an abandoned transaction always costs — server-side resources held until it elapses — and a short one costs you the whole walk.

The failure is TypeDB.Error with code TSV12 and kind :server, raised from inside whatever is consuming the stream. TypeDB.Error.retryable?/1 answers true for it, which is honest — but a retry restarts the walk from the first page, because the snapshot it was reading is gone.

Sort, or the pages will not line up

Paging is offset and limit, appended to your query as its last two stages. Those are only meaningful against a total order, so a query without a sort stage may hand you a row twice and never hand you another. The driver cannot check this for you: TypeQL is the server's language, and a driver that went looking for the word sort in your query would be wrong the first time someone sorted in a sub-pipeline.

Sort on something unique. sort $n where $n is a @key is the easy case.

What it does not do

fetch pipelines cannot be paged. offset and limit after a fetch stage are a syntax error — measured, 400 TQL0 — so this streams conceptRows and nothing else. A fetch query reaches you as the server's own parse error rather than as anything this driver invented.

It raises rather than returning {:error, _}. A lazy stream has no place to put an error tuple: the failure happens inside Enum.to_list/1, long after this function returned. Every failure is a TypeDB.Error, the same one query/4 would have returned, so rescue/try works as it does everywhere else. This is why there is no stream!/4 — there is nothing for it to do differently.

Options

Query and transaction options (see TypeDB.Options), plus:

  • :page_size — rows per request. Defaults to 1000. It is also used as the :answer_count_limit of each page unless you set that yourself, so a page can never be silently truncated.
  • :timeout — bounds each page's request, not the walk.
  • :deadline — the same, per request. Neither bounds the walk; the walk is bounded by :transaction_timeout_millis, above.

:transaction_type is not among them. A walk that wrote would be a walk whose pages moved underneath it.

transaction(conn, database, type, fun, opts \\ [])

@spec transaction(
  conn(),
  String.t(),
  TypeDB.Transaction.type(),
  (TypeDB.Transaction.t() -> result),
  keyword()
) :: result | {:error, TypeDB.Error.t()}
when result: term()

Runs fun inside a transaction, committing on success.

The transaction is committed when fun returns, rolled back when it returns {:error, _}, and rolled back before the exception propagates when it raises, throws or exits. :read transactions are closed rather than committed, since there is nothing to commit.

Returns whatever fun returned, except that a successful :write/:schema block whose commit fails returns {:error, %TypeDB.Error{}}.

Options

Transaction options from TypeDB.Options, plus :timeout and :deadline, which are forwarded to the request that opens the transaction. Anything else raises ArgumentError rather than being ignored — query options belong on the queries inside the block, not here.

Examples

TypeDB.transaction(conn, "social", :write, fn tx ->
  TypeDB.Transaction.query!(tx, "insert $p isa person, has name 'Alice';")
  TypeDB.Transaction.query!(tx, "insert $p isa person, has name 'Bob';")
  :ok
end)
#=> :ok

# Returning {:error, _} rolls back and returns that error unchanged.
TypeDB.transaction(conn, "social", :write, fn tx ->
  case TypeDB.Transaction.query(tx, "insert $p isa person;") do
    {:ok, _} -> {:error, :changed_my_mind}
    error -> error
  end
end)
#=> {:error, :changed_my_mind}

version(conn, opts \\ [])

@spec version(
  conn(),
  keyword()
) :: {:ok, TypeDB.Server.version()} | {:error, TypeDB.Error.t()}

Returns the server distribution and version. See TypeDB.Server.version/1.

version!(conn, opts \\ [])

@spec version!(
  conn(),
  keyword()
) :: TypeDB.Server.version()

Returns the server distribution and version, raising on failure. See TypeDB.Server.version!/1.