Mimir.Guard (Mimir v0.7.0)

Copy Markdown View Source

Turn-guard builders for a session loop's between-turn hook (RMA's turn_guard: option; 0.5.0+, hook shape unchanged through 0.10). Plain data in, plain verdict out — no RMA types.

for_grant/3 prices the session's accumulated usage with Mimir.Pricing and halts once the grant budget is spent — the control-plane soft half of enforcement, for runtimes where the gateway cannot sit in the data plane. caps/1 is the mimir-less form: plain cost/token/turn caps, no minted key.

A cost cap (the grant budget, or caps/1's :max_cost_microdollars) prices cache tokens too — cache_read_input_tokens and cache_creation_input_tokens are part of the usage map Guard prices through Mimir.Pricing, the same as input/output, because cost is cost. :max_total_tokens stays input + output only.

Guards never raise mid-run, but the two cost-check failures aren't the same outcome: a pricing-table miss (no rate anywhere for the model) degrades to :cont, leaving whatever caps remain to decide, and emits a [:mimir, :guard, :pricing_miss] telemetry warning once per process per model. A misconfigured pricing entry — Mimir.Pricing raising Mimir.Pricing.InvalidConfigError for an invalid rate or an unknown key — halts instead, with {:invalid_pricing, %{model:, usage:, message:}}, rather than letting the raise propagate: a bad config entry is a real problem the caller should stop and look at, not one the guard can shrug off the way it does a merely-unpriced model.

Summary

Functions

Build a guard from plain caps — the mimir-less form. Options

Build a guard from a route-response grant. Pass resp.grant from a %Mimir.RouteResponse{}. A %Grant{} with a nil budget_microdollars never halts on cost. opts take caps/1 options; caps are checked before the budget.

Types

guard_fun()

@type guard_fun() :: (turn_state() -> verdict())

turn_state()

@type turn_state() :: %{
  :usage => map(),
  :turns => non_neg_integer(),
  optional(any()) => any()
}

verdict()

@type verdict() :: :cont | {:halt, term()}

Functions

caps(opts \\ [])

@spec caps(keyword()) :: guard_fun()

Build a guard from plain caps — the mimir-less form. Options:

  • :max_turns — halt once turns reaches the cap
  • :max_total_tokens — halt once input+output tokens reach the cap
  • :max_cost_microdollars (with :model) — priced cost cap

Omitted caps don't constrain; with no options the guard always continues.

for_grant(grant, model, opts \\ [])

@spec for_grant(Mimir.Grant.t(), String.t(), keyword()) :: guard_fun()

Build a guard from a route-response grant. Pass resp.grant from a %Mimir.RouteResponse{}. A %Grant{} with a nil budget_microdollars never halts on cost. opts take caps/1 options; caps are checked before the budget.