Imp.Predict.ReActV2 (Imp v0.12.0)

Copy Markdown View Source

Native-tool-aware ReAct loop with structured history and typed completion.

ReActV2 preserves parallel tool call IDs and results in Imp.History and keeps unknown and failed tool calls as observations.

Which signatures get submit

A task signature with exactly one output of type :string and no constraints has an answer that the model can write as text, so its loop offers no submit tool. Every other signature (several outputs, one output that is not text, or one text output with constraints such as an enum, whose allowed values reach the model in submit's schema) gets the reserved submit tool, whose parameters are the signature's outputs, exactly as DSPy's ReActV2 has it. The name submit is reserved for every signature, so a user tool cannot take it.

The prediction

The prediction's fields are the signature's outputs and nothing else, so an output may be called anything, history included. How the turn went is in its metadata:

  • :history — the turn's full Imp.History, to pass back as the :history input of the next call.
  • :termination_reason — how the turn ended (below).
  • :termination_cause — present whenever the turn was interrupted: for :last_text, :forced_submit and :extracted, the interruption that led to the last request. For :incomplete, the same interruption, unless the turn's context window was full or its Imp.Deadline had passed, which would refuse any further request too and so are named instead; termination_error holds the errors of the requests that failed. One of :max_iters, :parse_error, :prediction_error, :empty_tool_calls, :context_window_exceeded and :deadline_exceeded.
  • :termination_error — for :incomplete, the redacted errors of the requests that failed.
  • :finished_by_tool — the terminal tool that ended the turn.
  • :unexecuted_tool_calls — tool calls the last request made that were not run.
  • :context_projection — how many prior episodes were left out of a request (see below).

Imp.Prediction.complete?/1 is false exactly when the reason is :incomplete.

How a turn ends

  • :answered (one unconstrained text output). A step that says something and calls no tool is the answer, in that one request.
  • :submit (every other signature). The model calls submit with the signature's outputs.
  • :finished_by_tool. finish_on maps a tool name to fn arguments, result, inputs -> {:finish, outputs} | :continue end. It runs after that tool's call executes, with the string-keyed arguments the tool received; {:finish, outputs} validates outputs against the signature exactly as a submit would and ends the turn, with finished_by_tool naming the tool. :continue leaves the loop running. When one step calls several terminal tools, the first in call order finishes the run; the rest still execute and are recorded, and a submit in the same step still wins. Outputs that fail validation are recorded as that call's result, the same error a bad submit records, and the loop continues.
  • :last_text, :forced_submit, :extracted. The turn was interrupted and its last request answered (below).
  • :incomplete. The turn was interrupted and has no answer: its last request was answered with no valid answer in it, or the turn ran out of time or context window.

A turn that could not get a model response returns {:error, %Imp.Predict.ReActV2.StepError{}} rather than a prediction: its last request failed with an Imp.LMError, a step's request was stopped by the provider's content filter (Imp.LMError with content_filtered: true; sending it again would be filtered and billed again, so no last request is made, as DSPy makes none after a step's LM error), its LM client raised ({:lm_failed, client, exception}), or a renderer raised ({:adapter_format_failed, adapter, exception}). The error's :reason is that request's error as Imp.Predict returns it, which Imp.Errors classifies through the struct, and its :history is the turn's history as far as it got, with every tool call that ran and its result. A step that fails with an Imp.OperationalSafetyError (a route, cost, transport or budget guard) ends the turn the same way at once, with no last request. A tool call whose outcome is unknown is not such a failure: it is an observation, and the turn goes on.

When a turn is interrupted

A turn is interrupted when it reaches max_iters, when a step's request fails (:prediction_error, :parse_error) other than by the content filter, or when a step of a signature with submit calls no tool (:empty_tool_calls). An enum output is such a signature: text that is not a submit call is :empty_tool_calls, never an answer, and a turn with no valid submit, the forced one included, ends :incomplete.

With one unconstrained text output, a step that calls no tool and says nothing is not an interruption: it is an empty answer, and the turn ends there (:answered). Saying nothing is how a model declines to answer, and asking it again would make declining cost a second request.

With one unconstrained text output, every interruption takes the same path: one more request, and its text is the answer (:last_text). The request is a step like any other: the same tools and the same tool_choice: "auto". A provider may refuse a history of tool calls when no tools are declared (Anthropic does), and a changed roster changes the prompt prefix a provider caches. It does not say tool_choice: "none": a model told that while it wants a tool can write the call as text in its own tool markup (seen from inkling through OpenRouter, and through DeepInfra), and that text would become the answer. If the model calls a tool, the call is not run; the completion's text, if any, is the answer, and the calls are kept in unexecuted_tool_calls. A completion that says nothing is an empty answer rather than an error. If the process's Imp.Deadline has already passed, no request is made and the turn is :incomplete with termination_cause: :deadline_exceeded.

With submit, an interruption forces one more request with tool_choice naming submit (:forced_submit), as DSPy does. If a provider cannot honor that tool contract, a tools-disabled typed extractor derives the task outputs from the original inputs and accumulated history (:extracted).

The last request says nothing about why it is being made unless :last_request_note is given: one line of host text put in front of it as a user message and kept in the returned history like any other turn. It is recorded as a step that said nothing and called nothing, so it renders as that user message alone, with no assistant reply after it, in the request and whenever the history is passed back. Imp writes no sentence of its own.

A request refused because the context window is full is not an interruption of this kind: a further request would be refused the same way, so the turn ends at once as :incomplete with termination_cause: :context_window_exceeded (see below).

For a single unconstrained text output, a step describes that task field directly, including its description. Other signatures use next_thought alongside the typed submit tool. The other step output is tool_calls (agent_tool_calls when the task text output itself is named tool_calls). The provider holds the tool roster natively. Plain text with no tool call fills the declared text field through the step signature's :text_field metadata. It completes the text task in one LM call and is appended to the history as its own step. A tool call the model writes as JSON rather than calling natively is accepted with tool for name and args or parameters for arguments (Imp.Adapter.Types.ToolCall); a map that names no tool at all is kept as a malformed-call observation.

So a step asks for one thing, whichever adapter formats it. The step has a tools input, the roster as text in DSPy's shape ("name, whose description is <desc>...</desc>. It takes arguments {...}.", the arguments being the schema's properties, required and $defs as JSON, the properties in the schema's order when it keeps one and by name otherwise), and a tool_calls output whose description shows how to write a call. When the LM calls tools natively (every LM unless its client says otherwise; Imp.Clients.ReqLLM asks the model registry), the tools are sent natively and Imp.Predict formats the step without both fields, in every adapter and in the JSON fallback: the model answers with tool calls or with text. When the LM cannot (the registry says the model has no tool calling, or the LM is Imp.Clients.TRLLM), no tools are sent: the step lists them in tools, the model writes its calls in tool_calls, and earlier steps are replayed as text (the step's fields, then the results), never as native tool messages. This is DSPy's structure. The guidance says where a text answer goes: in the task text field, with tool_calls left empty when that field is described. With native tool calls the Chat adapter asks for no field markers at all: its system message lays out the inputs alone and says the message without a tool call is the answer. A reply in plain text, or in markers, is read as that field by Imp.Adapter.Chat. Durable step history keeps next_thought and tool_calls for compatibility with saved conversations; adapters project them into the declared fields when rendering. A stored turn that carries a one-text-output task's answer and no step outputs is replayed as a step that answered in text. The name tools is reserved: a task signature cannot have a field of that name.

On a recognized context-window refusal, up to eight smaller requests omit oldest prior episodes from the prompt, preserving their full durable history. Completed signature outputs delimit episodes; a trailing unfinished prior group is kept together. Current-call tool observations are never omitted or replayed. Omission counts appear in :context_projection and native :context_projected events. If the current call and instructions alone exceed the window, an incomplete prediction retains history and context diagnostics. This is lossy prompt selection, not summarization or deletion of memory.

Tool history retains provider-native reasoning text and opaque reasoning details for continuation, including after Imp.History.dump/1 and Imp.History.load!/1. These are operational protocol data and must remain unmodified. Store history privately; use redacted events or Imp.History.redact/1 for diagnostic copies.

Summary

Functions

Builds a ReAct loop over tools for signature.

Types

t()

@type t() :: %Imp.Predict.ReActV2{
  finish_on: term(),
  last_request_note: term(),
  max_iters: term(),
  react: term(),
  signature: term(),
  tool_order: term(),
  tool_policy: term(),
  tools: term()
}

Functions

new(signature, tools, opts \\ [])

@spec new(Imp.Signature.t() | String.t(), [Imp.Tool.t()], keyword()) :: t()

Builds a ReAct loop over tools for signature.

tools is a list of Imp.Tool values; the name submit is reserved. The signature decides whether the loop has a submit tool (see the module documentation).

Options

  • :lm - The model each step calls. When absent, each call uses Imp.Settings' :lm.

  • :adapter - The adapter that renders each step's request and parses its reply. When absent, each call uses Imp.Settings' :adapter.

  • :demos (list of term/0) - Worked examples for the step predictor, as for Imp.Predict. The default value is [].

  • :config (keyword/0) - Request options for every step (temperature, max tokens, ...). The tool roster is added here, so do not pass :tools. The default value is [].

  • :adapter_opts (keyword/0) - Options handed to the adapter beside the loop's own guidance; a host injects its renderers here, such as Imp.Adapter.Chat's :system_renderer. The default value is [].

  • :metadata (map of term/0 keys and term/0 values) - Free-form metadata kept on the step predictor. The default value is %{}.

  • :max_iters (non_neg_integer/0) - Steps before the turn is interrupted. An interrupted turn ends with the forced submit, or, for a signature with one unconstrained text output, one last text-only request. The default value is 20.

  • :tool_policy - Which tool calls may run; see Imp.ToolPolicy. The default value is :allow.

  • :last_request_note - One line of host text put in front of the last request of an interrupted turn, as a user message, and kept in the history: the last text-only request for a signature with one unconstrained text output, the forced submit for every other. nil says nothing; Imp writes no sentence of its own. The default value is nil.

  • :finish_on - Tools whose call ends the turn: a map from tool name to fn arguments, result, inputs -> {:finish, outputs} | :continue end. See "How a turn ends" above. The default value is %{}.