Normandy.TurnError exception (normandy v2.2.0)

View Source

A turn that failed for a reason that is not the model, the input or a guardrail.

Turn.Server.run/2, Turn.Session.run/2 and AgentProcess.run/3 return {:error, exception}. The exception is Normandy's own (Normandy.LLMError, Normandy.Schema.ValidationError, Normandy.Guardrails.ViolationError, Normandy.HookHaltError, Normandy.ModelCatalogError) when that is what failed, and a TurnError otherwise.

AgentProcess differs by turn engine for any other raised exception (a bug, not a Normandy error). With turn_engine: :server it is replied as a :task_down TurnError. In inline mode (the default) run/3 and cast/3 reply that exception bare, as {:error, exception}, rather than wrapping it. In both modes the turn runs in a worker process: an exit or a throw in it, or the worker killed from outside, is a :task_down TurnError.

:type says which:

  • :handler_crashed - a handler or hook raised, exited or threw; :handler names it (:append, :convert, :validate, :guard, :emit, :admit, :after_tool, :before_llm, :after_compact, :on_session_start); :reason is the exception, or the raw exit reason or thrown term (normalized by Exception.normalize/3, so the kind is not kept)
  • :persist_failed - the session store rejected a write; :reason is its reason, or {:store_exit, exit_reason} when the store call exited (a call timeout, a dead store process): the exit reason with no stacktrace and a call's arguments replaced by their count, {:timeout, {GenServer, :call, 3}}. Turn.Session.delete/1 returns it when the store refused the delete or its call exited, with the reason reduced to its tags ({:store_exit, {:timeout, {GenServer, :call, "a list"}}})
  • :approval_timeout - nobody approved in time and on_approval_timeout: :fail; :reason is the list of tool call ids
  • :task_down - the LLM call, the compactor or the tool dispatch raised an exception that is not one of the above, or its worker stopped; :reason is the exception or the exit reason
  • :session_unavailable - the session could not be loaded or started (store, registry, template or credential failure); :reason is that failure. A session store's own reason is reduced to its tags: atoms and integers are kept, tuples are walked, an exception becomes its module, and any other term becomes a description of its type ({:query_failed, "a map"}). A load that raised, exited or threw gives the exception (an exit's or a throw's reason). Turn.Session.approve/2 returns it with reason: :no_session when no server is live; Turn.Session.delete/1 returns it when the registry cannot answer or the live server is not the supervisor's (reason: {:not_stopped, :not_found})
  • :queue_full - Normandy.Coordination.AgentProcess refused a run or cast because :max_pending_runs runs and casts were already queued or in flight; :reason is that limit. Nothing ran
  • :unexpected_event, :unresumable_state - a persisted turn that can't continue, or a Normandy bug; :reason describes it

A TurnError never carries a stacktrace; Normandy logs it where the failure happened. Its message names the failure but not values from it (they can hold the config or user data); read :reason for those.

:reason in replies, in the store and in telemetry

The TurnError replied to the caller holds the full :reason, a non-Normandy exception included, except a session store's reason (see :session_unavailable above). The copy Turn.Server persists as a failed turn's Turn.State.error, and every telemetry :exception reason, are redacted: a :reason that is an exception other than Normandy's own (a KeyError, a MatchError) is %{exception: KeyError}, because its fields and its message can hold the value it failed on, and a bare one is emitted as %TurnError{type: :task_down, reason: %{exception: KeyError}}. The same holds for such an exception at any depth of the tuples and lists in a reason. A call in an exit reason keeps its arity in place of its arguments, which hold the call's message: {%KeyError{}, {GenServer, :call, [server, message, 5000]}} is {%{exception: KeyError}, {GenServer, :call, 3}}. Normandy's own exceptions, and the rest of a reason (an exit reason's other terms, a thrown term, a store reason, tool call ids), are kept.

Summary

Types

t()

@type t() :: %Normandy.TurnError{
  __exception__: true,
  handler: atom() | nil,
  reason: term(),
  type: type()
}

type()

@type type() ::
  :handler_crashed
  | :persist_failed
  | :approval_timeout
  | :task_down
  | :session_unavailable
  | :queue_full
  | :unexpected_event
  | :unresumable_state