The notebook — notebook.db, a separate, portable SQLite store of user- and
LLM-governed memories, working and long-term: agent-defined schema, open SQL,
embeddings, backup and restore. Potentially precious (tutoring records,
curated knowledge); the node's obligation is durability — governance stays with
the user and the LLM. This module is its raw-SQL surface (via
YmerNode.Notebook.Repo).
Thin by design: the agent writes SQL; this module runs it. The only behaviour
beyond pass-through is the safety model and a _meta description layer.
Down, and serving
Two different facts answer "can the notebook take my statement now", and this module reports both rather than collapsing them. Down is a fact of the process: the repo does not resolve — its name is not registered, or it is registered and Ecto has not finished starting it, a window of microseconds — so nothing can reach the store at all. Serving is a fact of the store: a statement runs now.
A notebook serves when a statement runs now. The node answers serving from what the pool itself said — a failure it holds, or one query after a restart — never from process registration, which is the other fact, down. The two come apart: a repo restarted onto a store it cannot open is registered and not serving, and reporting it as either up or down would be a lie the caller acts on.
Every action reaches the repo through ensure_meta!/0, so every action runs
inside with_notebook/2. Its pre-check is
YmerNode.Notebook.Repo.running?/0 — a microsecond structural read, the name
and Ecto's own record of the repo behind it; its catch re-reads that, because
the repo can stop between the check and the query. Serving is never probed
ahead of an ordinary call. It is read only where a failure has already put the
evidence in hand, through YmerNode.Notebook.Repo.not_serving?/1, which is
why the happy path costs nothing.
The classification, whole, at the pre-check and again in the catch:
- the repo resolves and the call succeeded — served, and the value goes back untouched.
- resolves, and the pool could not provide a connection —
{:error, :notebook_not_serving}. The node cannot open the store; the causes left are permissions, disk faults and an edit under a running node, and a wait fixes none of them, so the answer carries no retry advice. The node does not wait either: the pool's own reconnect backoff runs from one second toward thirty, longer than an honest answer. - does not resolve, with the lock held for a restore —
{:error, :operation_in_progress}, at once. Only a restore stops the repo, so that window is expected and lasts the file swap; it is never waited on. - does not resolve, anything else — a bounded wait, because a supervised restart
is microseconds and answering inside one would be a terminal instruction for
a condition already resolved. A capture never stops the repo, so a
:backuphold beside a stopped repo is a crash rather than a window, and an unreachable lock says nothing about the repo; both wait. If the repo comes back: at the pre-check nothing has run, so the body simply runs; in the catch a:readbody runs again, and a:writebody does not —{:error, :notebook_restarted}, whose instruction is to check whether the statement landed, since it may already have committed and this store cannot be rebuilt. - still does not resolve when the wait ends —
{:error, :notebook_down}, the stuck state, whose message asks for the one thing that helps, a node restart. - resolves, and a fault that is none of the above — back to the caller with its original kind and stacktrace, so a genuine fault is never swallowed.
One residual rides that last line, named rather than closed. The repo is
resolved after the failure rather than while it happens, so a repo that
stopped and came back inside a single call — a supervised restart is
microseconds — resolves again by the time the catch reads it, and the failure
the stopped repo caused is classified as a fault of the caller's own statement
and re-raised.
What the repo raises when it does not resolve is a RuntimeError, or — inside
the start window — an ArgumentError out of the registry; each carries a
message and nothing structural to match, so recognising either would mean
reading a dependency's text — the one thing this classification does not do.
The cost is one raw exception in front of a caller whose call happened to race
a restart, and it closes itself: the next call finds the notebook serving.
The classification binds to structural state throughout: process
registration and Ecto's own record of the repo, the lock's held op, and — for
serving — an exception's documented type and reason field. Matching the text
of a dependency's message stays forbidden; that text is free to change, and a
check that read it would keep passing until the day the wording moved.
Read-only enforcement (non-obvious — do not remove)
query/1 runs the statement inside YmerNode.Notebook.Repo.transaction/1 and ALWAYS
rolls back. A SELECT's rows ride out through the rollback value; any write a
crafted statement performs is discarded. This is the hard guarantee — the
leading-keyword check is only a friendly router, and is leaky on its own because
WITH … DELETE is a valid write that starts with WITH. EXPLAIN reads also
route here; PRAGMA is rejected (some pragmas are connection-level writes a
rollback won't undo — use the tables/schema actions for introspection).
execute/1 blocks ATTACH/DETACH. The store is the only database worth
protecting, and the node database sits beside it — but ATTACH reaches any
SQLite file the process can open, not merely those two, so a prompt-injected
agent must not have the verb at all. VACUUM INTO is blocked
on the same ground — it writes a full copy of the database to any path the
process can write (bare VACUUM compaction stays legal). The execute path has NO
rollback backstop, so its guard is load-bearing on its own. The classifier
strips ALL SQL comments and any leading whitespace/(/; and case-folds, so
;ATTACH …, (ATTACH …), and an inline attach/**/database … cannot slip
past. Multi-statement tails are inert because
YmerNode.Notebook.Repo.query/1 prepares only the first statement (exqlite Sqlite3.prepare)
— this MUST NOT be swapped for Exqlite.Sqlite3.execute/2, which would run the tail.
Result shaping (exqlite command heuristic — do not trust it)
ecto_sqlite3 builds the query without a :command, so exqlite classifies a
statement by a case-sensitive substring match for INSERT/UPDATE/DELETE. That
drops the columns of a SELECT whose text contains such a substring, and reports a
wrong/stale changes() count for lowercase DML and substring-bearing DDL. So
query/1 passes command: :select to keep columns, and execute/1 derives
affected_rows from SELECT changes() on the same connection
(YmerNode.Notebook.Repo.checkout/2) keyed on our own leading-keyword classifier.
The vector layer
notebook.db loads sqlite-vec, so the agent can keep vec0 virtual tables
beside ordinary ones and run KNN search through query/1. The usage discipline
that comes with it — the INTEGER-PRIMARY-KEY join rule, the delete-from-both
rule, and sqlite-vec's requirement that MATCH+LIMIT run in its own
subquery/CTE — lives in YmerNode.Mcp.Tools.Notebook.Schemas' action notes, written
for the agent that composes the SQL; help {tool: "notebook"} returns them.
_meta
SQLite has no column comments, so ensure_meta!/0 keeps a _meta(target, description, …) table (Datasette-style table:name / column:table.col keys)
that tables/0 and table_schema/1 join against. It is created idempotently at
the start of each call (cheap; a no-op once present) BEFORE the query rollback
transaction, so a cold query cannot roll its creation away.
Summary
Functions
Runs a single DDL/DML statement (no result rows). Rejects ATTACH/DETACH,
and VACUUM INTO while allowing bare VACUUM.
Returns {:ok, %{affected_rows: n}} or {:error, reason}.
Runs a read-only SELECT/WITH/EXPLAIN statement inside an always-rolled-back
transaction. Returns {:ok, %{columns: [...], rows: [...]}} or {:error, reason}
(:not_read_only if the statement is not a read — PRAGMA is rejected; use the
tables/schema introspection actions instead). Columns are recovered via
command: :select regardless of exqlite's command heuristic.
Full schema for one table: %{table, virtual, description, columns, indexes}.
Regular tables use PRAGMA table_info/index_list; vec0 virtual tables parse
their CREATE VIRTUAL TABLE … vec0(…) DDL. Returns {:error, :table_not_found}
if the table does not exist.
Lists user tables (excludes _meta, sqlite_* internals, and sqlite-vec shadow
tables). Each entry: %{name, virtual, row_count, description}. Virtual (vec0)
tables report row_count: nil. Shadow tables are detected by name prefix
"<virtual_table>_" — keep companion tables outside that prefix (the documented
pattern names them confluence_pages next to vec_confluence_pages).
Runs fun and answers a notebook that cannot take it instead of raising at it.
Functions
Runs a single DDL/DML statement (no result rows). Rejects ATTACH/DETACH,
and VACUUM INTO while allowing bare VACUUM.
Returns {:ok, %{affected_rows: n}} or {:error, reason}.
affected_rows is the exact SELECT changes() count when the statement's leading
keyword is INSERT/UPDATE/DELETE, and 0 for everything else — DDL, and the
rare CTE-led WITH … DELETE write, reported conservatively as 0. It is derived from
our own comment-stripped, case-folded leading-keyword classifier plus
changes() on the
same connection, NOT from exqlite's result, whose command classification is an
unreliable case-sensitive substring match.
Runs a read-only SELECT/WITH/EXPLAIN statement inside an always-rolled-back
transaction. Returns {:ok, %{columns: [...], rows: [...]}} or {:error, reason}
(:not_read_only if the statement is not a read — PRAGMA is rejected; use the
tables/schema introspection actions instead). Columns are recovered via
command: :select regardless of exqlite's command heuristic.
Full schema for one table: %{table, virtual, description, columns, indexes}.
Regular tables use PRAGMA table_info/index_list; vec0 virtual tables parse
their CREATE VIRTUAL TABLE … vec0(…) DDL. Returns {:error, :table_not_found}
if the table does not exist.
Lists user tables (excludes _meta, sqlite_* internals, and sqlite-vec shadow
tables). Each entry: %{name, virtual, row_count, description}. Virtual (vec0)
tables report row_count: nil. Shadow tables are detected by name prefix
"<virtual_table>_" — keep companion tables outside that prefix (the documented
pattern names them confluence_pages next to vec_confluence_pages).
Answers {:error, reason} while the notebook is down or cannot open its store.
That is the one return here that is not a list, so match it before treating the
answer as one.
Runs fun and answers a notebook that cannot take it instead of raising at it.
body is what the caller is doing, not what the guard should do with it:
:read for the three actions that only read, :write for execute/1. The
guard owns the consequence — only a read may be run a second time, because a
write's statement may already have committed before the failure reached here,
and this is the one store the node cannot rebuild.
Every action above reaches the repo, so every action runs inside this guard. The pre-check answers the ordinary stopped case without raising at all; the catch covers the race where the repo stops between the check and the query, and both classify from structural state rather than from the exception's text, which belongs to a dependency. Anything caught while the repo is serving goes back exactly as it arrived — kind, value and stacktrace — so a genuine fault is never swallowed; the moduledoc names the residual that rides that rule, a failure raised while the repo was gone and read once the repo is back. The moduledoc's "Down, and serving" section owns the whole classification and what each answer means.
Public ONLY so both halves of that catch are directly testable: the four actions hand it a fixed body, so a test working through them can neither make one raise while the repo stays up nor stop the repo from inside one. No MCP exposure.