LemonAgent.Proxy (lemon_agent v0.1.0)

Proxy stream function for apps that route LLM calls through a server.

The server manages authentication and proxies requests to LLM providers. This module reconstructs partial messages from server-sent events (SSE) that have the partial field stripped to reduce bandwidth.

Usage

Use stream_proxy/3 as the stream_fn option when creating an Agent that needs to go through a proxy:

config = %AgentLoopConfig{
  model: model,
  stream_fn: fn model, context, opts ->
    LemonAgent.Proxy.stream_proxy(model, context, %ProxyStreamOptions{
      auth_token: get_auth_token(),
      proxy_url: "https://genai.example.com",
      temperature: opts.temperature,
      max_tokens: opts.max_tokens,
      reasoning: opts.reasoning
    })
  end,
  ...
}

Summary

Types

proxy_assistant_message_event()

Proxy event types - server sends these with partial field stripped to reduce bandwidth.

Functions

parse_streaming_json(json)

Parse partial/streaming JSON, attempting to complete incomplete JSON.

stream_proxy(model, context, options)

Stream function that proxies through a server instead of calling LLM providers directly.

Types

proxy_assistant_message_event()

@type proxy_assistant_message_event() ::
  {:start}
  | {:text_start, non_neg_integer()}
  | {:text_delta, non_neg_integer(), String.t()}
  | {:text_end, non_neg_integer(), String.t() | nil}
  | {:thinking_start, non_neg_integer()}
  | {:thinking_delta, non_neg_integer(), String.t()}
  | {:thinking_end, non_neg_integer(), String.t() | nil}
  | {:tool_call_start, non_neg_integer(), String.t(), String.t()}
  | {:tool_call_delta, non_neg_integer(), String.t()}
  | {:tool_call_end, non_neg_integer()}
  | {:done, :stop | :length | :tool_use, LemonAi.Types.Usage.t()}
  | {:error, :aborted | :error, String.t() | nil, LemonAi.Types.Usage.t()}

Proxy event types - server sends these with partial field stripped to reduce bandwidth.

Events:

{:start} - Stream started
{:text_start, content_index} - Text content block started
{:text_delta, content_index, delta} - Text content delta
{:text_end, content_index, content_signature} - Text content block ended
{:thinking_start, content_index} - Thinking content block started
{:thinking_delta, content_index, delta} - Thinking content delta
{:thinking_end, content_index, content_signature} - Thinking content block ended
{:tool_call_start, content_index, id, tool_name} - Tool call started
{:tool_call_delta, content_index, delta} - Tool call arguments delta (JSON)
{:tool_call_end, content_index} - Tool call ended
{:done, reason, usage} - Stream completed successfully
{:error, reason, error_message, usage} - Stream ended with error

Functions

parse_streaming_json(json)

@spec parse_streaming_json(String.t()) :: map()

Parse partial/streaming JSON, attempting to complete incomplete JSON.

Returns an empty map if parsing fails.

stream_proxy(model, context, options)

@spec stream_proxy(
  LemonAi.Types.Model.t(),
  LemonAi.Types.Context.t(),
  LemonAgent.Proxy.ProxyStreamOptions.t()
) :: LemonAi.EventStream.t()

Stream function that proxies through a server instead of calling LLM providers directly.

The server strips the partial field from delta events to reduce bandwidth. We reconstruct the partial message client-side.

Parameters

model - The AI model to use
context - The conversation context
options - ProxyStreamOptions with auth and proxy configuration

Returns

Returns an LemonAi.EventStream that emits events as they arrive from the proxy server.