Dllb.MetaAST.Ingest (Dllb v0.8.2)

Copy Markdown View Source

Enhanced batch ingestion pipeline for loading MetaAST data into dllb.

Bridges MetaAST analysis output to dllb's storage layer by producing batches of document and edge operations ready for bulk insertion. Supports incremental re-indexing via Dllb.MetaAST.Diff.

Mirrors the Rust dllb-code-intel::ingest module with the same deterministic ID scheme, enabling cross-language ID compatibility.

Usage

# Full ingest
batch = Dllb.MetaAST.Ingest.prepare_batch(tree, "lib/app.ex", "elixir")
queries = Dllb.MetaAST.Ingest.to_queries(batch)
Dllb.batch_transaction(queries)

# Incremental ingest (only changed entities)
diff = Dllb.MetaAST.Diff.diff_trees(old_tree, new_tree)
queries = Dllb.MetaAST.Ingest.incremental_queries(new_tree, diff, "lib/app.ex", "elixir")
Dllb.batch_transaction(queries)

Summary

Types

t()

Functions

incremental_queries(new_tree, diff, file_path, language)

Generate queries for incremental re-indexing based on a diff.

merge_batches(batches)

Merge multiple file batches into a single large batch for bulk loading.

prepare_batch(root, file_path, language)

Walk the MetaAST tree and produce a batch of dllb operations.

to_queries(batch)

Convert a batch into dllb query strings wrapped in BEGIN/END BATCH.

Types

t()

@type t() :: %Dllb.MetaAST.Ingest{
  documents: [map()],
  edges: [map()],
  file_path: String.t(),
  language: String.t(),
  stats: map()
}

Functions

incremental_queries(new_tree, diff, file_path, language)

@spec incremental_queries(
  tuple(),
  Dllb.MetaAST.Diff.diff_summary(),
  String.t(),
  String.t()
) :: [
  String.t()
]

Generate queries for incremental re-indexing based on a diff.

Only produces operations for changed entities:

  • DELETE + CREATE for modified/added entities
  • DELETE for removed entities

merge_batches(batches)

@spec merge_batches([t()]) :: t()

Merge multiple file batches into a single large batch for bulk loading.

prepare_batch(root, file_path, language)

@spec prepare_batch(tuple(), String.t(), String.t()) :: t()

Walk the MetaAST tree and produce a batch of dllb operations.

Produces:

  • A document for each Container and FunctionDef node
  • Containment edges: container -> contains -> function
  • Import edges: container -> imports -> (stable ID)
  • Call edges: function -> calls -> (stable ID)

to_queries(batch)

@spec to_queries(t()) :: [String.t()]

Convert a batch into dllb query strings wrapped in BEGIN/END BATCH.