Tools

LLM tools let models call functions in your application. The library handles tool definition encoding, streaming tool call chunks, automatic execution, and result injection.

Module tools

The recommended approach. Define a module implementing the LLM.Tool behaviour:

defmodule MyApp.Tools.ReadFile do
  @behaviour LLM.Tool

  @impl true
  def name, do: "read_file"

  @impl true
  def description, do: "Read the contents of a file at the given path"

  @impl true
  def input_schema do
    %{
      "type" => "object",
      "properties" => %{
        "path" => %{
          "type" => "string",
          "description" => "Absolute or relative file path"
        }
      },
      "required" => ["path"]
    }
  end

  @impl true
  def execute(%{"path" => path}, _context) do
    case File.read(path) do
      {:ok, content} -> {:ok, content}
      {:error, reason} -> {:error, "Could not read file: #{:format.error(reason)}"}
    end
  end
end

Callbacks

The execute/2 callback

Receives two arguments:

1. Input map — the decoded arguments from the model, matching your JSON Schema
2. Context map — currently contains %{messages: [...]} with the conversation history

def execute(%{"query" => query}, %{messages: messages}) do
  # Access conversation history if needed
  {:ok, "Results for: #{query}"}
end

Inline tools

For quick prototyping, create tools inline:

tool = LLM.Tool.new(%{
  name: "shell",
  description: "Execute a shell command",
  input_schema: %{
    "type" => "object",
    "properties" => %{
      "command" => %{"type" => "string"}
    },
    "required" => ["command"]
  },
  run: fn %{"command" => cmd} ->
    {output, _exit_code} = System.cmd("sh", ["-c", cmd])
    {:ok, output}
  end
})

Arity-1 vs arity-2 functions

The :run function can be arity 1 (just input) or arity 2 (input + context):

# Arity 1 — just input
LLM.Tool.new(%{
  name: "double",
  description: "Double a number",
  input_schema: %{
    "type" => "object",
    "properties" => %{"n" => %{"type" => "integer"}},
    "required" => ["n"]
  },
  run: fn %{"n" => n} -> {:ok, n * 2} end
})

# Arity 2 — input + context
LLM.Tool.new(%{
  name: "last_user_message",
  description: "Get the last user message",
  input_schema: %{"type" => "object", "properties" => %{}},
  run: fn _input, %{messages: messages} ->
    last = messages |> Enum.reverse() |> Enum.find(&(&1.role == :user))
    {:ok, last && last.content}
  end
})

Using tools

Pass tools to generate/2 or stream/2:

# With module tools
{:ok, response} = LLM.generate("Read the file mix.exs",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.Tools.ReadFile]
)

# With inline tools
{:ok, response} = LLM.generate("Double 21",
  provider: :openai,
  model: "gpt-4",
  tools: [double_tool]
)

# Mix of modules and inline tools
{:ok, response} = LLM.generate("Read mix.exs and double 21",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.Tools.ReadFile, double_tool]
)

Tool with extra context

The {module, extra_context} tuple merges extra data into the execution context:

defmodule MyApp.Tools.Search do
  @behaviour LLM.Tool

  @impl true
  def name, do: "search"

  @impl true
  def description, do: "Search the database"

  @impl true
  def input_schema do
    %{
      "type" => "object",
      "properties" => %{"query" => %{"type" => "string"}},
      "required" => ["query"]
    }
  end

  @impl true
  def execute(%{"query" => query}, context) do
    repo = Map.get(context, :repo, MyApp.Repo)
    results = repo.all(from p in "posts", where: ilike(p.title, "%#{query}%"))
    {:ok, inspect(results)}
  end
end

# Pass extra context
{:ok, response} = LLM.generate("Find posts about Elixir",
  provider: :openai,
  model: "gpt-4",
  tools: [{MyApp.Tools.Search, %{repo: MyApp.Repo}}]
)

Automatic vs manual tool execution

Automatic (default)

generate/2 and collect/2 auto-execute tool calls by default:

# Tools are executed automatically
{:ok, response} = LLM.generate("What's the weather in Tokyo?",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.Tools.Weather]
)
# response.message.content contains the final text after tool execution

Control the behavior:

# Disable auto-execution
{:ok, response} = LLM.generate("What's the weather?",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.Tools.Weather],
  auto_tools: false
)

# Limit rounds (default: 10)
{:ok, response} = LLM.generate("Complex task",
  provider: :openai,
  model: "gpt-4",
  tools: tools,
  max_rounds: 3
)

Manual execution

With auto_tools: false, tool call chunks are included in the collected response. You can process them yourself:

{:ok, response} = LLM.generate("Read mix.exs",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.Tools.ReadFile],
  auto_tools: false
)

# Check for tool calls in the message content
case response.message.content do
  content when is_list(content) ->
    tool_uses = Enum.filter(content, &match?({:tool_use, _, _, _}, &1))
    # Execute manually...

  _ ->
    :no_tool_calls
end

JSON Schema

Tool input schemas use standard JSON Schema:

%{
  "type" => "object",
  "properties" => %{
    "location" => %{
      "type" => "string",
      "description" => "City name"
    },
    "units" => %{
      "type" => "string",
      "enum" => ["celsius", "fahrenheit"],
      "description" => "Temperature units"
    }
  },
  "required" => ["location"]
}

See json-schema.org for the full specification.

Tool call anatomy

When a model decides to call a tool, the library produces a LLM.Stream.ToolCall struct with these fields:

The tool_call_id linkage

Each tool call has an id. When the tool is executed, the result must reference that id via tool_call_id:

Model generates tool call:   id: "call_abc123", name: "search", arguments: %{"q" => "elixir"}
Library executes search/2:   {:ok, "Elixir is a functional language..."}
Result message created:      role: :tool, tool_call_id: "call_abc123", name: "search", content: "Elixir is..."

This linkage is how providers match tool results to the correct call. Without it, the conversation breaks.

Streaming accumulation

Tool call arguments arrive incrementally during streaming. The library buffers them and only emits a ToolCall when the JSON is complete:

SSE event 1: arguments = "{\"q\":"      → no chunk emitted (JSON incomplete)
SSE event 2: arguments = "\"elixir\"}"   → JSON parses → %LLM.Stream.ToolCall{...}

You never see partial arguments.

Multiple simultaneous tool calls

When the model calls several tools at once, each gets a unique index:

[
  %LLM.Stream.ToolCall{id: "call_1", name: "get_weather", arguments: %{"city" => "NYC"}, index: 0, complete: true},
  %LLM.Stream.ToolCall{id: "call_2", name: "get_time", arguments: %{"tz" => "EST"}, index: 1, complete: true}
]

For deeper details on how tool calls flow through the full message lifecycle, see the Messages guide.

Next steps

• Messages, Roles, and Tool Calls — message structure, roles, and the tool call lifecycle
• Streaming — handle streaming responses with tools
• Configuration — HTTP client and runtime options