Messages, Roles, and Tool Calls

This guide explains how messages flow through the library, how roles work across providers, and how tool calls are represented and linked.

Multi-turn conversations

Every LLM request carries a list of messages. The simplest request sends a single user prompt:

{:ok, response} = LLM.generate("What is Elixir?",
  provider: :openai,
  model: "gpt-4"
)

Behind the scenes, this creates a LLM.Context with one message:

messages: [%LLM.Message{role: :user, content: "What is Elixir?"}]

For multi-turn conversations, pass previous messages via the :messages option:

alias LLM.Message

messages = [
  Message.new(:user, "What is Elixir?"),
  Message.new(:assistant, "Elixir is a functional language built on the BEAM VM."),
  Message.new(:user, "What is the BEAM?")
]

{:ok, response} = LLM.generate("Tell me more",
  provider: :openai,
  model: "gpt-4",
  messages: messages
)

The :messages list is prepended before the new prompt. The final message list sent to the provider is:

messages ++ [Message.new("Tell me more")]

Building context with LLM.Context

For full control, build a context directly:

alias LLM.Context

# From a keyword list
context = Context.new(
  system: "You are a helpful assistant",
  messages: [
    "What is Elixir?",
    Message.new(:assistant, "Elixir is a functional language."),
    "Tell me about GenServers"
  ]
)

{:ok, response} = LLM.generate(context, provider: :openai, model: "gpt-4")

The keyword list accepts:

• :system — system prompt string
• :messages — list of messages (strings become user messages automatically)
• :tools — list of tool modules or LLM.Tool structs
• :provider_state — provider-specific state (e.g., OpenAI Responses previous_response_id)

You can also pass a %LLM.Context{} struct directly as the prompt:

context = %LLM.Context{
  system: "You are a pirate",
  messages: [Message.new(:user, "Ahoy!")]
}

{:ok, response} = LLM.generate(context, provider: :openai, model: "gpt-4")

Message roles

Every message has a role that tells the provider who wrote it.

:user

Messages from the human user. This is the default role when creating a message from a string:

LLM.Message.new("Hello")
#=> %LLM.Message{role: :user, content: "Hello"}

:assistant

Responses from the model. When the model generates text, the result has role: :assistant.

An assistant message can have string content (plain text) or list content (mixed text, tool calls, thinking):

# Plain text response
%Message{role: :assistant, content: "The answer is 42."}

# Structured content with tool call
%Message{
  role: :assistant,
  content: [
    {:text, "Let me look that up."},
    {:tool_use, "call_abc123", "search", %{"query" => "elixir"}}
  ]
}

:tool

Results from tool executions. Each tool result message must include a :tool_call_id that matches the id from the tool call:

%Message{
  role: :tool,
  tool_call_id: "call_abc123",  # matches the ToolCall.id
  name: "search",               # matches the ToolCall.name
  content: "Elixir is a functional language..."
}

:system

System instructions. In most providers, this is set via the :system option rather than as a message:

LLM.generate("Hello", provider: :openai, model: "gpt-4",
  system: "You are a helpful assistant"
)

If you need a system message in the message list, use the :system option. The provider handles it appropriately:

• OpenAI — becomes the first message with "role": "system"
• Anthropic — sent as the top-level "system" field
• Gemini — sent as "systemInstruction"
• OpenAI Responses — sent as the "instructions" field

:developer

Developer-specific instructions (Anthropic). Anthropic extracts :developer messages and merges them into the system prompt:

messages = [
  Message.new(:developer, "Always respond in JSON format"),
  Message.new(:user, "What is 2+2?")
]

{:ok, response} = LLM.generate("What is 2+2?",
  provider: :anthropic,
  model: "claude-sonnet-4-5-20250514",
  messages: messages
)

Other providers pass :developer messages through as regular messages.

Role encoding per provider

Message content

Content can be a string or a list of tagged tuples.

String content

Simple text-only messages:

%Message{role: :user, content: "Hello"}
%Message{role: :assistant, content: "Hi there!"}

List content (structured parts)

When the model produces mixed output (text + tool calls + thinking), content is a list:

%Message{
  role: :assistant,
  content: [
    {:thinking, "Let me search for that..."},
    {:text, "Here's what I found:"},
    {:tool_use, "call_1", "search", %{"query" => "elixir"}}
  ]
}

The five content part types:

Tool call anatomy

When a model decides to call a tool, the library represents it as a LLM.Stream.ToolCall struct.

Fields

Example

%LLM.Stream.ToolCall{
  id: "call_abc123",
  name: "get_weather",
  arguments: %{"location" => "San Francisco", "units" => "celsius"},
  index: 0,
  complete: true
}

How tool_call_id links results to calls

The flow is:

1. Model generates a tool call with an id (e.g., "call_abc123")
2. The tool is executed and the result is captured
3. A result message is created with tool_call_id matching the call's id:

# Step 3: result message links back to the call
%Message{
  role: :tool,
  tool_call_id: "call_abc123",  # ← matches ToolCall.id
  name: "get_weather",
  content: "72°F, sunny"
}

This linkage is critical — providers use it to match tool results to the correct tool call. Without it, the provider cannot continue the conversation.

How arguments are accumulated during streaming

Tool call arguments often arrive in fragments. The library accumulates them internally:

SSE event 1: {"tool_calls": [{"id": "call_1", "function": {"name": "search", "arguments": "{\"q\":"}}]}
SSE event 2: {"tool_calls": [{"function": {"arguments": "\"hello\"}"}}]}

After accumulation, the library parses the complete JSON and emits a single ToolCall chunk:

%LLM.Stream.ToolCall{id: "call_1", name: "search", arguments: %{"q" => "hello"}, complete: true}

You never see partial arguments — only complete tool calls.

Multiple simultaneous tool calls

When the model calls multiple tools at once, each gets a unique index:

[
  %LLM.Stream.ToolCall{id: "call_1", name: "get_weather", arguments: %{"city" => "NYC"}, index: 0, complete: true},
  %LLM.Stream.ToolCall{id: "call_2", name: "get_time", arguments: %{"timezone" => "EST"}, index: 1, complete: true}
]

Tool call IDs per provider

Complete tool call sequence

Here's the full lifecycle of a tool call round:

Round 1:
  User: "What's the weather in Tokyo?"

  Model generates:
    assistant: "Let me check the weather."
    assistant: tool_call(get_weather, %{"city" => "Tokyo"})

  Library executes get_weather(%{"city" => "Tokyo"})
  Result: "25°C, sunny"

Round 2:
  Messages sent to provider:
    user:      "What's the weather in Tokyo?"
    assistant: ["Let me check the weather.", tool_use(call_1, get_weather, ...)]
    tool:      tool_result(call_1, "25°C, sunny")

  Model generates:
    assistant: "The weather in Tokyo is 25°C and sunny."

In code

# The full sequence is handled automatically:
{:ok, response} = LLM.generate("What's the weather in Tokyo?",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.WeatherTool]
)

# response.message.content is the final text after tool execution:
# "The weather in Tokyo is 25°C and sunny."

With manual control

{:ok, stream} = LLM.stream("What's the weather in Tokyo?",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.WeatherTool],
  auto_tools: false
)

{:ok, chunks, stream} = LLM.Stream.next(stream)

# Check for tool calls
tool_calls = Enum.filter(chunks, &match?(%LLM.Stream.ToolCall{complete: true}, &1))

# Execute manually, build result messages, start follow-up

Multi-turn with tools

A full conversation with tool calls involves multiple messages:

alias LLM.Message

messages = [
  # User asks
  Message.new(:user, "What's the weather in Tokyo?"),

  # Assistant responds with tool call
  %Message{
    role: :assistant,
    content: [
      {:text, "Let me check that for you."},
      {:tool_use, "call_1", "get_weather", %{"city" => "Tokyo"}}
    ]
  },

  # Tool result
  %Message{
    role: :tool,
    tool_call_id: "call_1",
    name: "get_weather",
    content: "25°C, sunny"
  },

  # Assistant's final response (after tool execution)
  Message.new(:assistant, "The weather in Tokyo is 25°C and sunny.")
]

# Continue the conversation
{:ok, response} = LLM.generate("And what about London?",
  provider: :openai,
  model: "gpt-4",
  messages: messages
)

Provider-specific differences

Anthropic

• :tool messages become "user" role with tool_result content blocks
• :assistant list content is encoded as typed content blocks (text, tool_use, thinking)
• :developer messages are extracted and merged into the system prompt
• Thinking blocks include a signature for verification

# Anthropic encodes this tool result as:
# %{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "call_1", "content": "..."}]}

%Message{role: :tool, tool_call_id: "call_1", content: "result"}

OpenAI Chat Completions

• :tool messages use "role": "tool" with tool_call_id
• :assistant list content separates text from tool_calls array
• Thinking uses reasoning_content in the delta

# OpenAI encodes this assistant message as:
# {"role": "assistant", "content": "Let me check", "tool_calls": [{"id": "call_1", "function": {...}}]}

%Message{
  role: :assistant,
  content: [
    {:text, "Let me check"},
    {:tool_use, "call_1", "search", %{"q" => "hello"}}
  ]
}

Gemini

• All roles except :assistant and :system map to "user"
• :assistant and :system map to "model"
• Tool calls are functionCall parts, results are functionResponse parts
• Gemini generates its own tool call IDs (gemini_ prefix)

OpenAI Responses

• Messages are encoded as items, not messages
• Tool calls become function_call items
• Tool results become function_call_output items
• Supports previous_response_id for conversation chaining

Next steps

• Tools — define and execute tools
• Streaming — work with streaming responses
• Providers — provider-specific details