Request Lifecycle And Concurrency

Copy Markdown View Source

You need concurrent requests without single-slot state overwrites.

After this guide, you will use request handles (ask/await) and collect multiple results safely.

Core Pattern

Jido.AI.Request follows a Task.async/await style model:

{:ok, req1} = MyApp.MathAgent.ask(pid, "2 + 2")
{:ok, req2} = MyApp.MathAgent.ask(pid, "3 + 3")

{:ok, r1} = MyApp.MathAgent.await(req1)
{:ok, r2} = MyApp.MathAgent.await(req2)

Per-request ReAct overrides travel with the request handle:

{:ok, req3} =
  MyApp.MathAgent.ask(pid, "Search only",
    allowed_tools: ["search"],
    tool_context: %{tenant_id: "acme"}
  )

Request Event Streaming

Use ask_stream/3 when a caller needs canonical ReAct runtime events while a request is running:

{:ok, %{request: request, events: events}} =
  MyApp.MathAgent.ask_stream(pid, "Show your work")

for event <- events do
  IO.inspect({event.kind, event.data})
end

{:ok, result} = MyApp.MathAgent.await(request)

The enumerable yields %Jido.AI.Reasoning.ReAct.Event{} values and stops after :request_completed, :request_failed, or :request_cancelled.

For mailbox-oriented integrations, pass a pid sink directly:

{:ok, request} =
  MyApp.MathAgent.ask(pid, "Show your work",
    stream_to: {:pid, self()}
  )

receive do
  {:jido_ai_request_event, %Jido.AI.Reasoning.ReAct.Event{} = event} ->
    IO.inspect(event.kind)
end

Pid sinks are request-scoped and use the calling process mailbox. They do not provide backpressure; keep handlers lightweight and always consume terminal events so request streams close cleanly.

Steering An Active ReAct Run

ask/await remains the request API. Mid-run steering is a separate control path:

{:ok, request} = MyApp.MathAgent.ask(pid, "Work on Q1")

{:ok, _agent} = MyApp.MathAgent.steer(pid, "Actually prioritize Q2", expected_request_id: request.id)

{:ok, result} = MyApp.MathAgent.await(request)

Use:

  • steer/3 for user-visible follow-up input on an active ReAct run
  • inject/3 for programmatic or inter-agent input on an active ReAct run

Important:

  • neither steer/3 nor inject/3 creates a new request handle
  • both reject idle agents with {:error, {:rejected, :idle}}
  • successful steer/3 / inject/3 means the input was queued, not durably persisted
  • if the run terminates before the runtime drains queued input, that input is dropped
  • normal concurrent ask/3 calls still busy-reject while a ReAct run is active
  • steering is ReAct-only in this version

Runtime Contract Map

Await Many

handles =
  ["2 + 2", "5 + 5", "8 + 8"]
  |> Enum.map(fn q -> elem(MyApp.MathAgent.ask(pid, q), 1) end)

results = Jido.AI.Request.await_many(handles, timeout: 30_000)
# [{:ok, ...}, {:ok, ...}, {:error, ...}]

Runtime End-To-End Snippet

alias Jido.AI.{Context, Turn}

{:ok, request} = MyApp.MathAgent.ask(pid, "What is 2 + 2?")

context =
  Context.new(system_prompt: "You are concise.")
  |> Context.append_user("What is 2 + 2?")

case MyApp.MathAgent.await(request, timeout: 15_000) do
  {:ok, result_text} ->
    turn = Turn.from_result_map(%{type: :final_answer, text: result_text})

    updated_context =
      context
      |> Context.append_assistant(turn.text)

    Context.to_messages(updated_context)

  {:error, {:rejected, :busy, message}} ->
    IO.puts("Request rejected: #{message}")

  {:error, :timeout} ->
    IO.puts("Request timed out")
end

Lifecycle States

Each request is tracked with status like:

  • :pending
  • :completed
  • :failed
  • :timeout

Agent state keeps request maps and compatibility fields (last_query, last_answer, etc.).

Completed request records may also include normalized meta when the runtime has it available. Common keys are:

  • :usage
  • :reasoning_details
  • :thinking_trace
  • :last_thinking

Example:

{:ok, status} = Jido.AgentServer.status(pid)
request_id = status.raw_state[:last_request_id]

get_in(status.raw_state, [:requests, request_id, :meta])
# %{usage: %{...}, reasoning_details: [...], ...}

The status snapshot separates the final assistant answer from completed tool outputs:

{:ok, status} = Jido.AgentServer.status(pid)

answer = status.snapshot.result
tool_results = status.snapshot.details[:tool_results] || []

tool_results is scoped to the current or most recent ReAct run and is meant for inspection. Persist durable business data from the tool itself, or return an allowed state effect when later turns need to read it.

Failure Mode: Timeouts Under Load

Symptom:

  • frequent {:error, :timeout} from await/2

Fix:

  • increase await timeout for expensive workloads
  • lower max_iterations for ReAct-style loops
  • reduce concurrency burst size or shard traffic

Defaults You Should Know

  • Default await timeout: 30_000ms
  • Default tracked request retention: 100 (evicts older entries)

When To Use / Not Use

Use this pattern when:

  • multiple caller processes can query the same agent
  • you need precise correlation from submission to result

Do not use this pattern when:

  • you only run single sequential calls and can tolerate sync wrappers

Next