Arcadic.Bulk (Arcadic v0.7.1)

Copy Markdown View Source

Bulk-create vertices and edges over ArcadeDB's POST /api/v1/batch/<db> NDJSON endpoint — the heavy-ingest sibling of Arcadic.Import.database, for records you hold in the client.

Each record is a caller map with ArcadeDB's structural keys — "@type" => "vertex" | "edge", "@class" => <TypeName>, arbitrary properties, and, on a vertex, an "@id" temporary-id string that edges reference. An edge carries "@from"/"@to" holding those vertex "@id" values (or an existing vertex's real #bucket:pos RID). Vertices must appear before the edges that reference them. Records are serialized to NDJSON and sent as one POST; the batch is structured record ingest, not statement text, so it is injection-inert (a value is stored verbatim). The 200 response's idMapping (temp "@id" → assigned RID) is surfaced as :id_mapping. Tenant-blind, HTTP-only.

Arcadic.Bulk.ingest(conn, [
  %{"@type" => "vertex", "@class" => "Person", "@id" => "p1", "name" => "Alice"},
  %{"@type" => "vertex", "@class" => "Person", "@id" => "p2", "name" => "Bob"},
  %{"@type" => "edge", "@class" => "Knows", "@from" => "p1", "@to" => "p2"}
])
#=> {:ok, %{vertices_created: 2, edges_created: 1, elapsed_ms: 7,
#=>        id_mapping: %{"p1" => "#1:0", "p2" => "#1:1"}}}

Operational contract (read before relying on it)

  • Atomic by default — any bad line rolls back the whole batch. Passing :commit_every forfeits that guarantee: a fault after a commit boundary leaves a committed prefix.
  • Create-only, not upsert — records are always created (no dedup; the vertex "@id" is an in-batch temporary handle for edge wiring, not a persisted identity — a re-ingest with the same "@id" creates fresh records). A lost response then a naive retry duplicates every vertex (the same non-confirmability class as Arcadic.command_async/4). For idempotent bulk-upsert use the UNWIND $rows idiom over Arcadic.command/4 (see usage-rules "Bulk loading").
  • One in-memory POST — the whole NDJSON body is held and sent at once, against one receive timeout. For a very large or streamed load, prefer Arcadic.Import.database (server-side fetch).
  • Line errors are reachable — a failed line's %Arcadic.Error{}.detail (quarantined from message/1/inspect/1 but reachable via error.detail) may contain the rejected record fragment (caller data — assume PII); redact before logging.

Summary

Functions

Bulk-create records (a list of caller vertex/edge maps). opts: :light_edges (bool), :commit_every (pos_integer), :timeout (ms). Returns {:ok, %{vertices_created, edges_created, elapsed_ms, id_mapping}} — where id_mapping maps each vertex "@id" temporary handle to its assigned real RID — or {:error, …} (:invalid_record on an unencodable record; :not_supported on a transport with no batch endpoint; else an Arcadic.Error/Arcadic.TransportError).

Bulk-create records, returning the counts map or raising.

Functions

ingest(conn, records, opts \\ [])

@spec ingest(Arcadic.Conn.t(), [map()], keyword()) ::
  {:ok,
   %{
     vertices_created: non_neg_integer(),
     edges_created: non_neg_integer(),
     elapsed_ms: non_neg_integer(),
     id_mapping: %{optional(String.t()) => String.t()}
   }}
  | {:error, :invalid_record | atom() | Exception.t()}

Bulk-create records (a list of caller vertex/edge maps). opts: :light_edges (bool), :commit_every (pos_integer), :timeout (ms). Returns {:ok, %{vertices_created, edges_created, elapsed_ms, id_mapping}} — where id_mapping maps each vertex "@id" temporary handle to its assigned real RID — or {:error, …} (:invalid_record on an unencodable record; :not_supported on a transport with no batch endpoint; else an Arcadic.Error/Arcadic.TransportError).

ingest!(conn, records, opts \\ [])

@spec ingest!(Arcadic.Conn.t(), [map()], keyword()) :: map()

Bulk-create records, returning the counts map or raising.