Behaviour for URL-specific content extraction strategies.

A strategy controls how a URL is fetched and how the response is converted to text for LLM consumption. Strategies are matched against URLs in order — the first strategy whose match?/2 returns true handles the request.

Callbacks

• match?/2 (required) — returns true if this strategy handles the given URI.
• request/2 (optional) — modifies the Req.Request before execution (URL rewriting, custom headers, etc.).
• extract/2 (required) — converts the Req.Response into a content string.

Example

defmodule MyApp.WikiStrategy do
  @behaviour Omni.Tools.WebFetch.Strategy

  @impl true
  def match?(uri, _opts), do: uri.host == "en.wikipedia.org"

  @impl true
  def request(req, _opts) do
    # Use the mobile API for cleaner content
    Req.merge(req, headers: [{"accept", "text/html"}])
  end

  @impl true
  def extract(response, _opts) do
    Html2Markdown.convert(response.body)
  end
end

Usage

Pass strategies to Omni.Tools.WebFetch.new/1:

Omni.Tools.WebFetch.new(
  strategies: [
    {MyApp.WikiStrategy, []},
    MyApp.AnotherStrategy
  ]
)