YmerNode.Notebook.Repo (Ymer Node v0.2.1)

Copy Markdown View Source

Ecto repo for notebook.db — the node's one deliberate local store.

This repo owns no schema: the LLM creates and evolves tables at runtime through the notebook MCP tool. There are no Ecto.Schema modules and no migrations, and this repo is deliberately absent from :ecto_repos — which lists YmerNode.Repo alone — so mix ecto.* never manages this file. Adding it to that list is what must never happen: it would put mix ecto.drop one keystroke from the one file the durability rule (YmerNode) exists for. The repo exists to give YmerNode.Notebook a supervised connection pool and to load the sqlite-vec vec0 extension on every connection.

On a fresh store those connections race each other to set the journal mode, and the first boot logs database is locked once. It is expected and is not fixed; README.md's layout section documents it for the operator who meets it in a container log.

Why the schema is not ours

Everything else the node serves is rebuildable: lose it and a resync or a reinstall restores it. notebook.db is the exception — it holds curated knowledge the user and the LLM govern between them, so the node's obligation is durability (YmerNode.Notebook.Backup), never schema authority. A repo that shipped migrations would claim the second.

Extension loading

The repo's init callback injects :load_extensions at boot rather than from config files, because the arch-specific absolute path resolves correctly in dev, test, and a release only once :ymer_node is loaded — which is true at repo start but not during compile-time config evaluation. exqlite disables the SQL-level load_extension() function after loading vec0, so callers cannot load further extensions.

The load is best-effort and silent: exqlite's extension loader runs SELECT load_extension(...) and discards the result, so a missing, corrupt, or wrong-arch binary would otherwise boot a healthy-looking repo with vec0 absent, failing only later as no such module: vec0. Two guards close that gap: the init callback File.exists?-checks the resolved, arch-suffixed binary and raises if absent (and the arch resolver raises for an unvendored arch), and YmerNode.Notebook.VecLoadCheck asserts SELECT vec_version() at boot wherever it is enabled.

Summary

Functions

Callback implementation for Ecto.Repo.all/2.

running?/0 after a bounded wait: true as soon as the repo can take a query, false when it still cannot at the end of the budget.

Callback implementation for Ecto.Repo.checked_out?/0.

Callback implementation for Ecto.Repo.checkout/2.

Callback implementation for Ecto.Repo.config/0.

The configured path of notebook.db, read straight from the application environment.

Callback implementation for Ecto.Repo.delete/2.

Callback implementation for Ecto.Repo.delete!/2.

Forces all connections in the repo pool to disconnect within the given interval.

Executes an EXPLAIN statement or similar for the given query according to its kind and the adapter in the given repository.

Callback implementation for Ecto.Repo.get/3.

Callback implementation for Ecto.Repo.get!/3.

Callback implementation for Ecto.Repo.get_dynamic_repo/0.

Callback implementation for Ecto.Repo.in_transaction?/0.

Callback implementation for Ecto.Repo.insert/2.

Callback implementation for Ecto.Repo.insert!/2.

Whether mode is one of the journal modes SQLite accepts — the bound journal_mode/0 enforces, as a guard, so the function that interpolates a mode into a PRAGMA refuses anything else at its own head rather than trusting its caller to have read the configuration. SQLite treats an unrecognised journal-mode value as a no-op that answers the current mode, so nothing downstream would refuse it.

The journal mode the pool is configured for — :journal_mode in the repo's application environment, :wal when unset, which is the adapter's own default.

Callback implementation for Ecto.Repo.load/2.

Whether failure is the pool's own report that it could not provide a connection — the serving half, and what "registered, and the store cannot be opened" looks like from a call site.

Callback implementation for Ecto.Repo.one/2.

Callback implementation for Ecto.Repo.one!/2.

Runs a custom SQL query.

Same as query/3 but returns result directly without :ok tuple and raises on invalid queries

Runs a custom SQL query that returns multiple results on the given repo.

Same as query_many/4 but returns result directly without :ok tuple and raises on invalid queries

Callback implementation for Ecto.Repo.reload/2.

Callback implementation for Ecto.Repo.rollback/1.

Whether the repo can take a query right now as a matter of process state — the liveness half of what a caller wants to know, and only that half.

Callback implementation for Ecto.Repo.start_link/1.

Callback implementation for Ecto.Repo.stop/1.

Callback implementation for Ecto.Repo.stream/2.

Converts the given query to SQL according to its kind and the adapter in the given repository.

Callback implementation for Ecto.Repo.update/2.

Callback implementation for Ecto.Repo.update!/2.

Functions

aggregate(queryable, aggregate, opts \\ [])

Callback implementation for Ecto.Repo.aggregate/3.

aggregate(queryable, aggregate, field, opts)

Callback implementation for Ecto.Repo.aggregate/4.

all(queryable, opts \\ [])

Callback implementation for Ecto.Repo.all/2.

all_by(queryable, clauses, opts \\ [])

Callback implementation for Ecto.Repo.all_by/3.

await_running()

running?/0 after a bounded wait: true as soon as the repo can take a query, false when it still cannot at the end of the budget.

A supervised restart of the repo lasts microseconds to a few milliseconds, and the pool's reconnect backoff starts at a second: the budget sits an order of magnitude above the first and well below the second, so a caller that lands in a restart is served rather than told the notebook is down, and a store the pool cannot open is never waited on. This is the one wait the node performs for a stopped repo. Polled, not subscribed, because the supervisor offers nothing to subscribe to and the poll is a microsecond read. YmerNode.Notebook's moduledoc owns how the answer combines with the rest.

checked_out?()

Callback implementation for Ecto.Repo.checked_out?/0.

checkout(fun, opts \\ [])

Callback implementation for Ecto.Repo.checkout/2.

child_spec(opts)

config()

Callback implementation for Ecto.Repo.config/0.

database_path()

The configured path of notebook.db, read straight from the application environment.

YmerNode.Notebook.Backup reads this at the start of a restore — after the id resolves and before the safety capture — and carries the value into the down window, where the repo child is deliberately stopped to swap the file underneath it. config/0 would answer too (it is a pure read plus the init/2 callback), but that callback runs vec0_path/0's filesystem checks and can raise — machinery a restore has no use for. This read is deliberately thinner, and owning it here keeps the value's single owner the module it describes, instead of a second Application.get_env in the caller.

default_options(operation)

Callback implementation for Ecto.Repo.default_options/1.

delete(struct, opts \\ [])

Callback implementation for Ecto.Repo.delete/2.

delete!(struct, opts \\ [])

Callback implementation for Ecto.Repo.delete!/2.

delete_all(queryable, opts \\ [])

Callback implementation for Ecto.Repo.delete_all/2.

disconnect_all(interval, opts \\ [])

@spec disconnect_all(non_neg_integer(), opts :: Keyword.t()) :: :ok

Forces all connections in the repo pool to disconnect within the given interval.

Once this function is called, the pool will disconnect all of its connections as they are checked in or as they are pinged. Checked in and idle connections will be randomly disconnected within the given time interval.

If the connection has a backoff configured (which is the case by default), disconnecting means an attempt at a new connection will be done immediately after, without starting a new process for each connection. However, if backoff has been disabled, the connection process will terminate. In such cases, disconnecting all connections may cause the pool supervisor to restart depending on the max_restarts/max_seconds configuration of the pool, so you will want to set those carefully.

If you need connections to be restarted periodically, set the :max_lifetime option in your repository configuration instead.

exists?(queryable, opts \\ [])

Callback implementation for Ecto.Repo.exists?/2.

explain(operation, queryable, opts \\ [])

@spec explain(
  :all | :update_all | :delete_all,
  Ecto.Queryable.t(),
  opts :: Keyword.t()
) ::
  String.t() | Exception.t() | [map()]

Executes an EXPLAIN statement or similar for the given query according to its kind and the adapter in the given repository.

Examples

# Postgres
iex> MyRepo.explain(:all, Post)
"Seq Scan on posts p0  (cost=0.00..12.12 rows=1 width=443)"

iex> Ecto.Adapters.SQL.explain(Repo, :all, Post)
"Seq Scan on posts p0  (cost=0.00..12.12 rows=1 width=443)"

# MySQL
iex> MyRepo.explain(:all, from(p in Post, where: p.title == "title")) |> IO.puts()
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | p0    | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    1 |    100.0 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+

# Shared opts
iex> MyRepo.explain(:all, Post, analyze: true, timeout: 20_000)
"Seq Scan on posts p0  (cost=0.00..11.70 rows=170 width=443) (actual time=0.013..0.013 rows=0 loops=1)\nPlanning Time: 0.031 ms\nExecution Time: 0.021 ms"

It's safe to execute it for updates and deletes, no data change will be committed:

iex> MyRepo.explain(Repo, :update_all, from(p in Post, update: [set: [title: "new title"]]))
"Update on posts p0  (cost=0.00..11.70 rows=170 width=449)\n  ->  Seq Scan on posts p0  (cost=0.00..11.70 rows=170 width=449)"

This function is also available under the repository with name explain:

iex> MyRepo.explain(:all, from(p in Post, where: p.title == "title"))
"Seq Scan on posts p0  (cost=0.00..12.12 rows=1 width=443)\n  Filter: ((title)::text = 'title'::text)"

Options

Built-in adapters support passing opts to the EXPLAIN statement according to the following:

AdapterSupported opts
Postgrexanalyze, verbose, costs, settings, buffers, timing, summary, format, plan, wrap_in_transaction
MyXQLformat, wrap_in_transaction

All options except format are boolean valued and default to false.

The allowed format values are :map, :yaml, and :text:

  • :map is the deserialized JSON encoding.
  • :yaml and :text return the result as a string.

The built-in adapters support the following formats:

  • Postgrex: :map, :yaml and :text
  • MyXQL: :map and :text

The wrap_in_transaction option is a boolean that controls whether the command is run inside of a transaction that is rolled back. This is useful when, for example, you'd like to use analyze: true on an update or delete query without modifying data. Defaults to true.

The :plan option in Postgrex can take the values :custom or :fallback_generic. When :custom is specified, the explain plan generated will consider the specific values of the query parameters that are supplied. When using :fallback_generic, the specific values of the query parameters will be ignored. :fallback_generic does not use PostgreSQL's built-in support for a generic explain plan (available as of PostgreSQL 16), but instead uses a special implementation that works for PostgreSQL versions 12 and above. Defaults to :custom.

Any other value passed to opts will be forwarded to the underlying adapter query function, including shared Repo options such as :timeout. Non built-in adapters may have specific behaviour and you should consult their documentation for more details.

For version compatibility, please check your database's documentation:

get(queryable, id, opts \\ [])

Callback implementation for Ecto.Repo.get/3.

get!(queryable, id, opts \\ [])

Callback implementation for Ecto.Repo.get!/3.

get_by(queryable, clauses, opts \\ [])

Callback implementation for Ecto.Repo.get_by/3.

get_by!(queryable, clauses, opts \\ [])

Callback implementation for Ecto.Repo.get_by!/3.

get_dynamic_repo()

Callback implementation for Ecto.Repo.get_dynamic_repo/0.

in_transaction?()

Callback implementation for Ecto.Repo.in_transaction?/0.

insert(struct, opts \\ [])

Callback implementation for Ecto.Repo.insert/2.

insert!(struct, opts \\ [])

Callback implementation for Ecto.Repo.insert!/2.

insert_all(schema_or_source, entries, opts \\ [])

Callback implementation for Ecto.Repo.insert_all/3.

insert_or_update(changeset, opts \\ [])

Callback implementation for Ecto.Repo.insert_or_update/2.

insert_or_update!(changeset, opts \\ [])

Callback implementation for Ecto.Repo.insert_or_update!/2.

is_journal_mode(mode)

(macro)

Whether mode is one of the journal modes SQLite accepts — the bound journal_mode/0 enforces, as a guard, so the function that interpolates a mode into a PRAGMA refuses anything else at its own head rather than trusting its caller to have read the configuration. SQLite treats an unrecognised journal-mode value as a no-op that answers the current mode, so nothing downstream would refuse it.

journal_mode()

The journal mode the pool is configured for — :journal_mode in the repo's application environment, :wal when unset, which is the adapter's own default.

YmerNode.Notebook.Backup switches a restored copy to this mode before the pool sees it, so the pool's first connections meet a file already in the mode they will set and have nothing to convert. Reading it here keeps the value's single owner the module it describes, the same reason database_path/0 exists. Raises ArgumentError for a value outside the modes SQLite accepts, so an unbounded value never reaches a PRAGMA.

load(schema_or_types, data)

Callback implementation for Ecto.Repo.load/2.

not_serving?(arg1)

Whether failure is the pool's own report that it could not provide a connection — the serving half, and what "registered, and the store cannot be opened" looks like from a call site.

The read is structural, like running?/0's: it matches DBConnection.ConnectionError's documented type and its documented reason field, never the message string, which belongs to a dependency and is free to change. :queue_timeout is set at exactly the two places the pool drops a request from its queue, both before any connection is handed out — so a client that timed out while holding a connection it did get, which is a slow statement and not a store that will not open, can never wear it. An Exqlite.Error — a malformed page, a SQL fault — is not this and stays a fault.

Three arrival shapes, because the same fact reaches three call sites differently: raised bare out of transaction/1 or checkout/2, wrapped in a {:badmatch, …} by an {:ok, _} = match on query/2's answer, and returned as query/2's own error tuple. The middle one is the raw term a catch clause is handed; rescue would normalise it to a MatchError struct, and the guard that reads this uses catch, because it has exits to contain as well.

One over-approximation the node accepts: the pool answers the same way when every connection is merely busy for longer than its queue window. This node serves one client on loopback over short statements, so saturation is not a live cause of it.

one(queryable, opts \\ [])

Callback implementation for Ecto.Repo.one/2.

one!(queryable, opts \\ [])

Callback implementation for Ecto.Repo.one!/2.

preload(struct_or_structs_or_nil, preloads, opts \\ [])

Callback implementation for Ecto.Repo.preload/3.

prepare_query(operation, query, opts)

Callback implementation for Ecto.Repo.prepare_query/3.

prepare_transaction(fun_or_multi, opts)

Callback implementation for Ecto.Repo.prepare_transaction/2.

put_dynamic_repo(dynamic)

Callback implementation for Ecto.Repo.put_dynamic_repo/1.

query(sql, params \\ [], opts \\ [])

Runs a custom SQL query.

If the query was successful, it will return an :ok tuple containing a map with at least two keys:

  • :num_rows - the number of rows affected
  • :rows - the result set as a list. nil may be returned instead of the list if the command does not yield any row as result (but still yields the number of affected rows, like a delete command without returning would)

Options

  • :log - When false, does not log the query
  • :timeout - Execute request timeout, accepts: :infinity (default: 15000);

Examples

iex> MyRepo.query("SELECT $1::integer + $2", [40, 2])
{:ok, %{rows: [[42]], num_rows: 1}}

iex> Ecto.Adapters.SQL.query(MyRepo, "SELECT $1::integer + $2", [40, 2])
{:ok, %{rows: [[42]], num_rows: 1}}

query!(sql, params \\ [], opts \\ [])

Same as query/3 but returns result directly without :ok tuple and raises on invalid queries

query_many(sql, params \\ [], opts \\ [])

@spec query_many(iodata(), Ecto.Adapters.SQL.query_params(), Keyword.t()) ::
  {:ok, [Ecto.Adapters.SQL.query_result()]} | {:error, Exception.t()}

Runs a custom SQL query that returns multiple results on the given repo.

In case of success, it must return an :ok tuple containing a list of maps with at least two keys:

  • :num_rows - the number of rows affected

  • :rows - the result set as a list. nil may be returned instead of the list if the command does not yield any row as result (but still yields the number of affected rows, like a delete command without returning would)

Options

  • :log - When false, does not log the query
  • :timeout - Execute request timeout, accepts: :infinity (default: 15000);

Examples

iex> MyRepo.query_many("SELECT $1; SELECT $2;", [40, 2])
{:ok, [%{rows: [[40]], num_rows: 1}, %{rows: [[2]], num_rows: 1}]}

iex> Ecto.Adapters.SQL.query_many(MyRepo, "SELECT $1; SELECT $2;", [40, 2])
{:ok, [%{rows: [[40]], num_rows: 1}, %{rows: [[2]], num_rows: 1}]}

query_many!(sql, params \\ [], opts \\ [])

Same as query_many/4 but returns result directly without :ok tuple and raises on invalid queries

reload(queryable, opts \\ [])

Callback implementation for Ecto.Repo.reload/2.

reload!(queryable, opts \\ [])

Callback implementation for Ecto.Repo.reload!/2.

rollback(value)

@spec rollback(term()) :: no_return()

Callback implementation for Ecto.Repo.rollback/1.

running?()

Whether the repo can take a query right now as a matter of process state — the liveness half of what a caller wants to know, and only that half.

A structural read, and a two-part one: the name the supervisor registers this repo under must resolve, and Ecto must hold the repo's adapter metadata, which it records only once the repo's start has finished. Ecto.Adapter.lookup_meta/1 answers both at once — it is the first thing every query resolves — and any failure to resolve is the answer; the message it may carry is never read. That text belongs to a dependency and is free to change: a check that read it would keep passing until the day the wording moved, and then quietly report a stopped notebook as running.

Registration alone is not enough, and measured so. A named repo's process is registered before its init/1 runs, and Ecto writes its metadata after the connection pool has started, so a read of the name alone admits a window — a median of 92 µs and up to 4.1 ms over 200 supervised restarts — in which every query raises a raw ArgumentError out of the registry. all_running/0 cannot close it either: the entry a dead repo left behind still names this module for a moment, and did so in 46 of those 200 restarts. Resolving the metadata for the pid the name points at is what makes the read exact.

It answers whether the process is there, never whether the store behind it can be opened: a repo restarted onto a store it cannot open is registered and not serving, and not_serving?/1 is the half that says so. YmerNode.Notebook's moduledoc owns how the two combine into one answer.

Three modules read it, and one wait stands behind two of them. YmerNode.Notebook reads it at the pre-check and again in the catch, because a SQL call takes no lock and can land anywhere inside another operation's down window; unless a restore's hold already explains the stop, a false from either goes on to await_running/0 below, so a repo inside a supervised restart is served rather than reported. YmerNode.Notebook.Backup calls await_running/0 outright before a capture, whose VACUUM INTO has nothing to read from a stopped repo — there the wait's own loop is what reads this function. And the test helper reads it once at boot, to put the sandbox in manual mode only when the repo started.

One read inside the guard does not wait, deliberately — the classification of a retry's own failure; YmerNode.Notebook's moduledoc owns that rule.

start_link(opts \\ [])

Callback implementation for Ecto.Repo.start_link/1.

stop(timeout \\ 5000)

Callback implementation for Ecto.Repo.stop/1.

stream(queryable, opts \\ [])

Callback implementation for Ecto.Repo.stream/2.

to_sql(operation, queryable, opts \\ [])

@spec to_sql(:all | :update_all | :delete_all, Ecto.Queryable.t(), Keyword.t()) ::
  {String.t(), Ecto.Adapters.SQL.query_params()}

Converts the given query to SQL according to its kind and the adapter in the given repository.

Examples

The examples below are meant for reference. Each adapter will return a different result:

iex> MyRepo.to_sql(:all, Post)
{"SELECT p.id, p.title, p.inserted_at, p.created_at FROM posts as p", []}

iex> MyRepo.to_sql(:update_all, from(p in Post, update: [set: [title: ^"hello"]]))
{"UPDATE posts AS p SET title = $1", ["hello"]}

iex> Ecto.Adapters.SQL.to_sql(:all, MyRepo, Post)
{"SELECT p.id, p.title, p.inserted_at, p.created_at FROM posts as p", []}

transact(fun_or_multi, opts \\ [])

Callback implementation for Ecto.Repo.transact/2.

transaction(fun_or_multi, opts \\ [])

Callback implementation for Ecto.Repo.transaction/2.

update(struct, opts \\ [])

Callback implementation for Ecto.Repo.update/2.

update!(struct, opts \\ [])

Callback implementation for Ecto.Repo.update!/2.

update_all(queryable, updates, opts \\ [])

Callback implementation for Ecto.Repo.update_all/3.