Mimir.Ingest (Mimir v0.4.0)

Copy Markdown View Source

Decision-correlated ingestion of raw agent-session events into Mimir.TurnEvents. Targets RMA's documented plain-data event maps — raw provider events plus the synthetic "rma.text_delta" — but accepts any binary-keyed %{"type" => ...} map. No RMA types.

Build a context with new/1 or from_route/2, then call handle_event/2 from your session handler's event hook. Each recognized raw map is promoted to a Mimir.Event (domain :llm — RMA telemetry ingested through this path is model-turn-shaped; agent-session-lifecycle framing belongs to the gateway Collector's agent.* path, not this one) and appended to the buffer keyed by request_id — the same buffer the embedder drains (Mimir.TurnEvents.take/1) when it meters the run.

metadata's "workflow_id"/"step_id" keys are unchanged from before — they now thread straight into the ingested event's typed workflow_id/ step_id fields (same names, promoted to Mimir.Event's correlation spine) instead of being merged into a loose payload map. decision_id has no dedicated Mimir.Event field, so it rides in the event's raw carve-out, alongside the original provider-native payload.

Classification

Raw frames are matched by structure, not by an open-ended list of provider type strings:

  • A map carrying a string "name" (the tool-use family — tool_use, custom_tool_use, server_tool_use, mcp_tool_use, ... — whatever the provider calls it, id+name is the shape) promotes to Event.llm(:tool_call, tool: %{id: ..., name: ...}, ...). "id" rides through as-is, including nil — matching the retired Mimir.TurnEvents.GenAI.tool_use/1's tolerance for a missing call id.
  • A map carrying integer "input_tokens"/"output_tokens" promotes to Event.llm(:usage, usage: %{input_tokens: ..., output_tokens: ...}, ...).
  • The synthetic "rma.text_delta" promotes to Event.llm(:turn_complete, ...) — recognized, but the closed llm vocabulary has no dedicated shape for a text delta, the same posture Mimir.Event.OTel documents for request_start/request_stop/turn_complete/exception: no fixed attribute shape, raw rendered verbatim at the export edge.
  • Anything else with a binary "type" is genuinely unrecognized: no event is emitted (never a placeholder type), a [:mimir, :ingest, :unknown_event] telemetry count fires, and the drop is logged at :debug.
  • Anything without a usable "type" (missing, or not a binary) is silently skipped, as before.

Every recognized event's raw carve-out keeps the original map verbatim (minus "type", which is stamped back in as "raw_type") — classification never discards the provider-native payload, it only promotes what it recognizes into typed commons alongside it.

Summary

Functions

Build a context straight from a Mimir.RouteResponse.

Ingest one raw event map. Recognized shapes are promoted to a Mimir.Event and appended to the buffer under ctx.request_id; genuinely unrecognized shapes are counted and dropped (see the moduledoc's "Classification" section). Always returns :ok; never raises into the session loop.

Options: :request_id (required), :decision_id, :metadata — a binary-keyed map. "workflow_id"/"step_id" correlate into the ingested event's typed ids; any other key rides in raw.

Types

t()

@type t() :: %Mimir.Ingest{
  decision_id: String.t() | nil,
  metadata: %{optional(String.t()) => term()},
  request_id: String.t()
}

Functions

from_route(resp, request_id)

@spec from_route(Mimir.RouteResponse.t(), String.t()) :: t()

Build a context straight from a Mimir.RouteResponse.

handle_event(ctx, event)

@spec handle_event(t(), map()) :: :ok

Ingest one raw event map. Recognized shapes are promoted to a Mimir.Event and appended to the buffer under ctx.request_id; genuinely unrecognized shapes are counted and dropped (see the moduledoc's "Classification" section). Always returns :ok; never raises into the session loop.

new(opts)

@spec new(keyword()) :: t()

Options: :request_id (required), :decision_id, :metadata — a binary-keyed map. "workflow_id"/"step_id" correlate into the ingested event's typed ids; any other key rides in raw.