GraphRAG with Arcadic

Setup

GraphRAG augments retrieval-augmented generation with a knowledge graph: entities and their relationships give a retriever a second axis (multi-hop traversal) alongside similarity search. This notebook builds a tiny graph — people, documents, and the edges between them — then walks every retrieval primitive Arcadic ships: bulk graph ingest, idempotent upsert, dense and sparse vector indexes, full-text search, hybrid fusion, INT8-quantized vectors, and a Cypher traversal.

This notebook needs a running ArcadeDB. The quickest way is Docker:

docker run -d --name arcadic-graphrag \
  -p 2480:2480 \
  -e JAVA_OPTS="-Darcadedb.server.rootPassword=playwithdata" \
  arcadedata/arcadedb:latest

Only the HTTP port (2480) is needed — every operation below (Bulk, FullText, Vector, Param) is HTTP-only; none of it runs over Bolt.

Every connection value can be overridden with an environment variable set before you launch Livebook (ARCADIC_HTTP_URL, ARCADIC_PASSWORD); otherwise the defaults match the docker run above.

Re-running cells. Most cells re-run cleanly: the raw CREATE VERTEX/EDGE TYPE and CREATE PROPERTY statements carry IF NOT EXISTS guards, the dense-vector and full-text index helpers emit IF NOT EXISTS internally, and the upsert, search, fuse, and traversal cells are idempotent or read-only. Two cells are not re-run-safe. The "Build a small knowledge graph" cell (Section 1) has guarded type-creates but then calls create-only Arcadic.Bulk.ingest/3, so re-running it adds a second copy of every vertex and edge. The "Dense + sparse vector indexes" cell (Section 3) raises already exists on re-run, because the LSM_SPARSE_VECTOR index has no IF NOT EXISTS form (create_sparse_index! takes no such flag). To redo either, run the Cleanup cell at the bottom, then run all cells top to bottom.

Install

Requires arcadic ≥ 0.5. Bulk, FullText, Vector.fuse/3, and Param are not yet in a published release, so this notebook errors under a ~> 0.4 pin. To run it today, point the install at the repo instead — {:arcadic, github: "baselabs/arcadic"} (or a local path: dep). Once 0.5 publishes, the ~> 0.5 pin below resolves to it and this note no longer applies.

Mix.install([
  {:arcadic, "~> 0.7"},
  {:kino, "~> 0.14"}
])

Connect

We create a throwaway graphrag_quickstart database to play in (only if it does not already exist, so this cell is safe to re-run).

http_url = System.get_env("ARCADIC_HTTP_URL", "http://localhost:2480")
password = System.get_env("ARCADIC_PASSWORD", "playwithdata")
database = "graphrag_quickstart"

admin = Arcadic.connect(http_url, database, auth: {"root", password})
{:ok, true} = Arcadic.Server.ready?(admin)

unless match?({:ok, true}, Arcadic.Server.database_exists?(admin, database)) do
  Arcadic.Server.create_database!(admin, database)
end

conn = Arcadic.connect(http_url, database, auth: {"root", password})

1. Build a small knowledge graph

Arcadic.Bulk.ingest/3 bulk-creates vertices and edges in one POST to ArcadeDB's batch endpoint — structured record ingest, not statement text, so it is injection-inert. It does not create schema, so the vertex/edge types must already exist.

Each vertex record carries an "@id" — a string that is only a temporary handle for wiring edges within this one batch; it is never persisted as a property. Each edge record's "@from"/"@to" reference those "@id" values. Because "@id" isn't persisted, we also give each vertex a real "id" property — the stable business key we'll MATCH/MERGE on in later cells, after the batch handles are long gone. The 200 response's id_mapping maps every temporary "@id" to the real RID ArcadeDB assigned it.

Arcadic.command!(conn, "CREATE VERTEX TYPE Person IF NOT EXISTS", %{}, language: "sql")
Arcadic.command!(conn, "CREATE VERTEX TYPE Document IF NOT EXISTS", %{}, language: "sql")
Arcadic.command!(conn, "CREATE EDGE TYPE KNOWS IF NOT EXISTS", %{}, language: "sql")
Arcadic.command!(conn, "CREATE EDGE TYPE WROTE IF NOT EXISTS", %{}, language: "sql")

{:ok, counts} =
  Arcadic.Bulk.ingest(conn, [
    %{
      "@type" => "vertex",
      "@class" => "Person",
      "@id" => "alice",
      "id" => "alice",
      "name" => "Alice",
      "role" => "engineer"
    },
    %{
      "@type" => "vertex",
      "@class" => "Person",
      "@id" => "bob",
      "id" => "bob",
      "name" => "Bob",
      "role" => "researcher"
    },
    %{
      "@type" => "vertex",
      "@class" => "Document",
      "@id" => "doc1",
      "id" => "doc1",
      "title" => "Graph databases 101",
      "body" => "A graph database models data as nodes and edges for fast multi-hop traversal."
    },
    %{
      "@type" => "vertex",
      "@class" => "Document",
      "@id" => "doc2",
      "id" => "doc2",
      "title" => "Vector search primer",
      "body" => "Vector search finds nearest neighbors in embedding space for semantic retrieval."
    },
    %{
      "@type" => "vertex",
      "@class" => "Document",
      "@id" => "doc3",
      "id" => "doc3",
      "title" => "Fraud detection with graphs",
      "body" => "Fraud rings can be uncovered by traversing the graph of transactions."
    },
    %{"@type" => "edge", "@class" => "KNOWS", "@from" => "alice", "@to" => "bob"},
    %{"@type" => "edge", "@class" => "WROTE", "@from" => "alice", "@to" => "doc1"},
    %{"@type" => "edge", "@class" => "WROTE", "@from" => "alice", "@to" => "doc3"},
    %{"@type" => "edge", "@class" => "WROTE", "@from" => "bob", "@to" => "doc2"}
  ])

{counts.vertices_created, counts.edges_created}

counts.id_mapping
|> Enum.map(fn {temp_id, rid} -> %{"temp_id" => temp_id, "rid" => rid} end)
|> Kino.DataTable.new()

2. Batched upsert (UNWIND)

Bulk.ingest is fast but create-only — a retry after a lost response duplicates every record. For an idempotent batch write, use the UNWIND $rows idiom over Arcadic.command/4 instead: one MERGE per unwound row, matched on the real id property (not a batch-scoped "@id"). This updates Alice's role and adds a new person, Carol, who was not part of the original bulk load.

rows = [
  %{"id" => "alice", "props" => %{"role" => "staff engineer"}},
  %{"id" => "carol", "props" => %{"name" => "Carol", "role" => "PM"}}
]

merge = "UNWIND $rows AS r MERGE (p:Person {id: r.id}) SET p += r.props RETURN count(p) AS c"

{:ok, [%{"c" => 2}]} = Arcadic.command(conn, merge, %{"rows" => rows})

# Replay the identical batch — MERGE matches the existing people instead of duplicating
# them, which is exactly what Bulk.ingest above would NOT have done.
{:ok, [%{"c" => 2}]} = Arcadic.command(conn, merge, %{"rows" => rows})

Arcadic.query!(conn, "MATCH (p:Person) RETURN count(p) AS total", %{})

3. Dense + sparse vector indexes

Arcadic.Vector.create_dense_index!/5 builds an LSM_VECTOR index for nearest-neighbor search over a float embedding. create_sparse_index!/5 builds an LSM_SPARSE_VECTOR index over a (tokens, weights) pair for a learned-sparse / BM25-style search.

Ordering matters for the sparse index: unlike dense (and unlike FullText, next section), a sparse index does not retro-index rows that already exist — only rows written after the index is created become searchable. We create both indexes before setting any vector data on the Document rows the bulk step above already created.

Arcadic.command!(conn, "CREATE PROPERTY Document.embedding IF NOT EXISTS ARRAY_OF_FLOATS", %{}, language: "sql")
Arcadic.command!(conn, "CREATE PROPERTY Document.tokens IF NOT EXISTS ARRAY_OF_INTEGERS", %{}, language: "sql")
Arcadic.command!(conn, "CREATE PROPERTY Document.weights IF NOT EXISTS ARRAY_OF_FLOATS", %{}, language: "sql")

:ok = Arcadic.Vector.create_dense_index!(conn, "Document", "embedding", 3, similarity: :cosine)

# create-before-load — see the caveat above.
:ok = Arcadic.Vector.create_sparse_index!(conn, "Document", "tokens", "weights")

doc_vectors = [
  %{
    "id" => "doc1",
    "embedding" => [1.0, 0.0, 0.0],
    "tokens" => [10, 11, 12],
    "weights" => [0.9, 0.5, 0.2]
  },
  %{
    "id" => "doc2",
    "embedding" => [0.0, 0.0, 1.0],
    "tokens" => [20, 21, 22],
    "weights" => [0.7, 0.4, 0.3]
  },
  %{
    "id" => "doc3",
    "embedding" => [0.9, 0.1, 0.0],
    "tokens" => [10, 13, 14],
    "weights" => [0.6, 0.5, 0.4]
  }
]

Arcadic.command!(
  conn,
  "UNWIND $rows AS r MATCH (d:Document {id: r.id}) " <>
    "SET d.embedding = r.embedding, d.tokens = r.tokens, d.weights = r.weights",
  %{"rows" => doc_vectors}
)

4. Full-text index + search

Arcadic.FullText.create_index/4 builds a Lucene-backed FULL_TEXT index. Unlike the sparse vector index above, FULL_TEXT does retro-index existing rows — the three Documents the bulk step created are searchable immediately, no reload required.

search/5 with with_score: true projects the BM25 relevance score alongside each row.

Arcadic.command!(conn, "CREATE PROPERTY Document.body IF NOT EXISTS STRING", %{}, language: "sql")

:ok = Arcadic.FullText.create_index(conn, "Document", "body")

{:ok, rows} =
  Arcadic.FullText.search(conn, "Document", "body", "graph", with_score: true, limit: 5)

Kino.DataTable.new(rows)

5. Hybrid retrieval (fuse)

Arcadic.Vector.fuse/3 runs several heterogeneous neighbor subqueries — dense, sparse, and/or full-text — and returns one fused, re-ranked list (reciprocal-rank fusion by default). Here we combine all three modalities: a dense arm (nearest to doc1's embedding direction), a sparse arm (nearest to doc1's learned-sparse (tokens, weights) signature, using the values loaded in Section 3), and a full-text arm ("graph"). doc1 and doc3 match all three signals and rank highest; doc2 matches only the dense arm and trails.

{:ok, rows} =
  Arcadic.Vector.fuse(conn, [
    {"Document", "embedding", [1.0, 0.0, 0.0], 3},
    {:sparse, "Document", "tokens", "weights", [10, 11, 12], [0.9, 0.5, 0.2], 3},
    {:fulltext, "Document", "body", "graph", 5}
  ])

Kino.DataTable.new(rows)

6. INT8-quantized ingest

Arcadic.Param.int8/1 wraps a list of signed bytes (-128..127) as ArcadeDB's $int8 typed-param marker, decoded server-side into a BINARY property — a compact way to store a quantized embedding signature. Requires ArcadeDB ≥ 26.5.1; HTTP-only (the decode lives in ArcadeDB's HTTP command handler — inert over Bolt).

Arcadic.command!(conn, "CREATE PROPERTY Document.signature IF NOT EXISTS BINARY", %{}, language: "sql")

Arcadic.command!(
  conn,
  "UPDATE Document SET signature = :sig WHERE id = :id",
  %{"sig" => Arcadic.Param.int8([0, 64, 127, -1, -128]), "id" => "doc1"},
  language: "sql"
)

Arcadic.query!(conn, "SELECT signature FROM Document WHERE id = :id", %{"id" => "doc1"},
  language: "sql"
)

7. Graph traversal (multi-hop retrieval)

The graph half of graphRAG: instead of (or alongside) similarity search, hop across relationships to pull in context. Starting from Alice, follow KNOWS to a contact, then WROTE to that contact's documents — "documents written by people Alice knows."

Arcadic.query!(
  conn,
  "MATCH (a:Person {id: $start})-[:KNOWS]->(friend:Person)-[:WROTE]->(d:Document) " <>
    "RETURN friend.name AS author, d.title AS title",
  %{"start" => "alice"}
)

Cleanup

Drop the throwaway database when you are done (or before a fresh top-to-bottom run).

Arcadic.Server.drop_database(admin, database)

← Previous Page Getting started with Arcadic

Next Page → Operations with Arcadic