Provider for any endpoint speaking the OpenAI chat-completions dialect.

Covers self-hosted vLLM (including behind Modal), OpenRouter, Together, Groq, and anything else exposing POST {base_url}/chat/completions.

Chat only. Embeddings need a task vocabulary, and "any OpenAI-compatible endpoint" cannot have one - see ExAgent.Providers.JinaV5 for the shape an embeddings provider takes.

Modal (vLLM behind Modal's proxy auth)

provider = ExAgent.Providers.OpenAICompatible.new(
  base_url: System.fetch_env!("MODAL_QWEN_URL") <> "/v1",
  model: "Qwen/Qwen3-VL-8B-Instruct",
  headers: [
    {"Modal-Key", System.fetch_env!("MODAL_KEY")},
    {"Modal-Secret", System.fetch_env!("MODAL_SECRET")}
  ],
  modalities: [:text, :image, :video]
)

{:ok, agent} = ExAgent.start_agent(provider: provider)

OpenRouter / Together / Groq (bearer auth)

provider = ExAgent.Providers.OpenAICompatible.new(
  base_url: "https://openrouter.ai/api/v1",
  model: "meta-llama/llama-3.3-70b-instruct",
  api_key: System.fetch_env!("OPENROUTER_API_KEY"),
  modalities: [:text, :image]
)

:api_key is sugar for an Authorization: Bearer header. :headers are merged afterwards, so an explicit header always wins.

Declaring modalities

:modalities describes this deployment, not the module: one vLLM container serves one model, and Qwen3-VL handles video where a text-only Llama container does not. It defaults to [:text], so a misconfigured deployment fails loudly rather than firing video at a model that cannot read it.

All four media modalities are declarable - :image, :video, :audio, and :document. Documents are shaped as the dialect's file content part (file_data for bytes, file_url for a URL), which is what a gateway fronting a document-reading model expects:

OpenAICompatible.new(
  base_url: "https://openrouter.ai/api/v1",
  model: "anthropic/claude-sonnet-4.5",
  api_key: key,
  modalities: [:text, :image, :document]
)

Declare only what the served model actually reads: nothing here is validated against the endpoint until the request is made.

No Files API

Compatible endpoints have no upload endpoint, so bytes are always sent as data: URIs. Past :max_inline_bytes (32 MB by default) the call returns {:error, %ExAgent.Error{type: :unsupported}} telling you to host the asset at a URL the container can reach - it is never silently truncated or retried.

Timeouts

:receive_timeout (5 minutes by default) is the ceiling for a single call, and a call site can lower it per request:

ExAgent.chat(agent, "Describe this clip",
  files: [%{path: "clip.mp4"}],
  receive_timeout: :timer.seconds(150))

Set it below any supervising timeout you wrap the call in - a Task killed at 180 s around a request still willing to wait 300 s reports a hung call as a dropped result. The connect timeout follows the same value.

Verifying the served model

Container swaps where config still names the old model are a common self-host failure. probe/1 checks:

case ExAgent.Providers.OpenAICompatible.probe(provider) do
  :ok -> :ready
  {:error, error} -> Logger.warning(Exception.message(error))
end