Jido.Messaging.Streaming (Jido Messaging v1.0.0)

Copy Markdown View Source

Streaming response support for progressive message updates.

Enables LLM-style streaming responses where message content is updated incrementally as it's generated. This is particularly useful for long-running agent responses.

Channel Support

  • Telegram: Uses editMessageText to update message content
  • Other channels can implement the StreamingChannel behaviour

Usage

# Start a streaming response
{:ok, stream} = Streaming.start(messaging_module, room, channel, chat_id, "Thinking...")

# Update the content as it streams in
:ok = Streaming.update(stream, "Thinking... Processing your request")
:ok = Streaming.update(stream, "Thinking... Processing your request. Here's what I found:")

# Finalize when complete
{:ok, final_message} = Streaming.finish(stream, "Here's what I found: [full response]")

Rate Limiting

Updates are automatically rate-limited to avoid hitting API limits. By default, updates are throttled to at most one every 100ms.

Summary

Types

t()

Functions

cancel(stream)

Cancel the streaming response without sending a final update.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

finish(stream, final_content)

Finalize the streaming response.

get_state(stream)

Get the current state of the stream.

schema()

Returns the Zoi schema

start(messaging_module, room, channel, chat_id, initial_content, opts \\ [])

Start a streaming response.

update(stream, content)

Update the streaming message content.

Types

t()

@type t() :: %Jido.Messaging.Streaming{
  channel: any(),
  chat_id: any(),
  current_content: nil | nil | binary(),
  last_update_at: nil | nil | integer(),
  message_id: nil | nil | any(),
  messaging_module: any(),
  min_update_interval_ms: integer(),
  pending_update: nil | nil | binary(),
  room: %Jido.Chat.Room{
    external_bindings: term(),
    id: term(),
    inserted_at: term(),
    metadata: term(),
    name: term(),
    type: term()
  }
}

Functions

cancel(stream)

Cancel the streaming response without sending a final update.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

finish(stream, final_content)

Finalize the streaming response.

Sends the final content and stops the stream process. Returns the final message info.

get_state(stream)

Get the current state of the stream.

schema()

Returns the Zoi schema

start(messaging_module, room, channel, chat_id, initial_content, opts \\ [])

Start a streaming response.

Sends an initial message and returns a stream handle for updates.

Options

  • :min_update_interval_ms - Minimum time between updates (default: 100)
  • :parse_mode - Telegram parse mode ("Markdown", "HTML", etc.)

update(stream, content)

Update the streaming message content.

If updates come faster than the rate limit, only the latest content will be sent when the throttle window expires.