Provider-agnostic LLM client: one API for 27 providers.

This module is the whole public surface. stream/3 and complete/3 send a LemonAi.Types.Context to any provider's model and return either a lazy LemonAi.EventStream of events or the final LemonAi.Types.AssistantMessage. Model configuration, cost accounting, per-provider circuit breaking and rate limiting, and context handoffs between providers are all handled behind this interface.

Usage

# Create a context
context = LemonAi.new_context(system_prompt: "You are a helpful assistant")
context = LemonAi.Types.Context.add_user_message(context, "Hello!")

# Stream a response
{:ok, stream} = LemonAi.stream(model, context)

for event <- LemonAi.EventStream.events(stream) do
  case event do
    {:text_delta, _idx, delta, _partial} -> IO.write(delta)
    {:done, _reason, message} -> IO.puts("\nDone!")
    _ -> :ok
  end
end

# Or get a complete response
{:ok, message} = LemonAi.complete(model, context)

Models

Models are defined with their provider, API type, and capabilities:

model = %LemonAi.Types.Model{
  id: "claude-sonnet-4-20250514",
  name: "Claude Sonnet 4",
  api: :anthropic_messages,
  provider: :anthropic,
  base_url: "https://api.anthropic.com",
  reasoning: true,
  input: [:text, :image],
  cost: %{input: 3.0, output: 15.0, cache_read: 0.3, cache_write: 3.75},
  context_window: 200_000,
  max_tokens: 64_000
}