Gralkor.GraphitiPool (jido_gralkor v4.1.0)

Copy Markdown View Source

Per-group Graphiti instance cache, plus the gateway for graphiti operations.

Holds one shared AsyncFalkorDB (the embedded redis-server child lives here) and lazily constructs one Graphiti instance per group_id. Cached in ETS for concurrent reads — for/1 only hits the GenServer on a cache miss (i.e. the first time any caller asks for a given group). Once cached, thousands of callers can read the instance simultaneously without going through the GenServer.

This is intentional. The spike (pythonx-spike/LEARNINGS.md) showed that Pythonx releases the GIL during graphiti's awaited I/O, so concurrent Elixir callers parallelise naturally. Serialising calls through a single GenServer would throw that away.

See ex-graphiti-pool in gralkor/TEST_TREES.md.

Summary

Functions

Ingest one episode (text content) into group_id via graphiti's add_episode. Auto-generates name and idempotency_key. When ontology is a module declared with use Gralkor.Ontology, its payload is materialised into graphiti's entity_types, edge_types, edge_type_map, and excluded_entity_types (cached per ontology module in the GenServer state).

Build communities for group_id.

Build indices and constraints across the whole graph.

Returns a specification to start this module under a supervisor.

Return the Graphiti instance for group_id, creating it on first use.

Pure projection from an __ontology__/0 payload to the plain data handed across the Pythonx boundary. A graphiti add_episode kwarg (entity_types, edge_types, edge_type_map, excluded_entity_types) is populated iff its payload collection is present; the Pythonx side never re-decides inclusion, it materialises exactly what this spec carries. No Pythonx, no LLM — this is the deterministic contract the materialisation half trusts.

Remove an episode and its orphaned edges/nodes from the graph.

Run graphiti's hybrid EDGE search against group_id. Returns {:ok, [%{fact:, created_at:, valid_at:, invalid_at:, expired_at:}]} ready for Gralkor.Format.format_facts/1.

Node search against group_id via graphiti's search_ with the NODE_HYBRID_SEARCH_RRF recipe. Unlike search/4 (which returns edges and whose node_labels filter matches edges by endpoint), this returns nodes — the right primitive for retrieving custom-entity nodes like Learning, which are not reliably reachable through edge search.

Build a compact one-line reason from a Pythonx.Error.

Functions

add_episode(server \\ __MODULE__, group_id, content, source_description, ontology, opts \\ [])

@spec add_episode(
  GenServer.server(),
  String.t(),
  String.t(),
  String.t(),
  module() | nil,
  keyword()
) ::
  :ok | {:error, term()}

Ingest one episode (text content) into group_id via graphiti's add_episode. Auto-generates name and idempotency_key. When ontology is a module declared with use Gralkor.Ontology, its payload is materialised into graphiti's entity_types, edge_types, edge_type_map, and excluded_entity_types (cached per ontology module in the GenServer state).

Options

  • :uuid — optional episode UUID forwarded to graphiti's add_episode. When given, graphiti fetches the existing episode and re-runs extraction against it (update path). When nil (default), graphiti generates a new UUID.

build_communities(server \\ __MODULE__, group_id)

@spec build_communities(GenServer.server(), String.t()) ::
  {:ok, %{communities: non_neg_integer(), edges: non_neg_integer()}}
  | {:error, term()}

Build communities for group_id.

build_indices(server \\ __MODULE__)

@spec build_indices(GenServer.server()) ::
  {:ok, %{status: String.t()}} | {:error, term()}

Build indices and constraints across the whole graph.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

for(server \\ __MODULE__, group_id)

@spec for(GenServer.server(), String.t()) :: any()

Return the Graphiti instance for group_id, creating it on first use.

Concurrent callers do not block each other once the instance is cached. Construction itself is serialised through the GenServer so two callers asking for the same group_id at the same time don't both construct it.

graphiti_boundary_spec(map)

@spec graphiti_boundary_spec(map()) :: %{optional(atom()) => term()}

Pure projection from an __ontology__/0 payload to the plain data handed across the Pythonx boundary. A graphiti add_episode kwarg (entity_types, edge_types, edge_type_map, excluded_entity_types) is populated iff its payload collection is present; the Pythonx side never re-decides inclusion, it materialises exactly what this spec carries. No Pythonx, no LLM — this is the deterministic contract the materialisation half trusts.

remove_episode(server \\ __MODULE__, group_id, episode_uuid)

@spec remove_episode(GenServer.server(), String.t(), String.t()) ::
  :ok | {:error, term()}

Remove an episode and its orphaned edges/nodes from the graph.

Calls graphiti's remove_episode(uuid) which deletes the episode, its entity edges that were created by that episode, and any entity nodes referenced only by the deleted episode.

search(server \\ __MODULE__, group_id, query, max_results)

@spec search(GenServer.server(), String.t(), String.t(), pos_integer()) ::
  {:ok, [map()]} | {:error, term()}

Run graphiti's hybrid EDGE search against group_id. Returns {:ok, [%{fact:, created_at:, valid_at:, invalid_at:, expired_at:}]} ready for Gralkor.Format.format_facts/1.

For retrieving custom-entity nodes (e.g. Learning for ERL recall) use search_nodes/5 — edge search's node-label filtering matches edges by endpoint and misses standalone nodes.

search_nodes(server \\ __MODULE__, group_id, query, max_results, opts \\ [])

@spec search_nodes(
  GenServer.server(),
  String.t(),
  String.t(),
  pos_integer(),
  keyword()
) ::
  {:ok, [map()]} | {:error, term()}

Node search against group_id via graphiti's search_ with the NODE_HYBRID_SEARCH_RRF recipe. Unlike search/4 (which returns edges and whose node_labels filter matches edges by endpoint), this returns nodes — the right primitive for retrieving custom-entity nodes like Learning, which are not reliably reachable through edge search.

Returns {:ok, [%{name:, summary:, attributes:}]} ordered by relevance.

Options

  • :node_labels — optional [String.t()]. When present, a SearchFilters(node_labels: …) restricts results to nodes carrying one of those labels (e.g. ["Learning"] for ERL recall). When absent, all nodes are eligible.

start_link(opts \\ [])

summarise_python_error(error)

@spec summarise_python_error(Pythonx.Error.t()) :: String.t()

Build a compact one-line reason from a Pythonx.Error.

Exception.message/1 on a Pythonx.Error joins the entire Python traceback (every frame, every embedding vector echoed in a call line) into one multi-line blob. On a transient FalkorDB reset that blob — and the whole embedding search vector inside it — gets dumped to the log via the rescue's {:error, {:python, reason}}.

The struct's :lines field is the output of Python's traceback.format_exception(type, value, traceback): a list whose final non-blank entry is the "ExceptionClass: message" summary line. We take that one line — the error's class and message — and drop the frames entirely.