Inference

inference provides reusable Elixir contracts for semantic model inference: requests, responses, clients, adapters, capabilities, trace metadata, redaction, and adapter conformance tests.

It is intentionally small. The package gives application code one stable shape for prompts, responses, client configuration, trace summaries, and adapter contracts. Provider SDKs, local agent runtimes, governed execution systems, and transport stacks remain outside the core contract.

It is not an Execution Plane wrapper. Execution Plane remains the lower runtime substrate. inference is the product-facing provider/model boundary used by standalone libraries such as trinity_coordinator and gepa_ex.

Installation

Add :inference to the application that wants the shared contract:

def deps do
  [
    {:inference, "~> 0.1.0"}
  ]
end

Provider-specific dependencies are opt-in. For example:

def deps do
  [
    {:inference, "~> 0.1.0"},
    {:gemini_ex, "..."},
    {:agent_session_manager, "..."}
  ]
end

The initial package ships adapter modules, not separate adapter packages:

Provider-specific dependencies are opt-in dependencies in the consuming application. GeminiEx is the direct Gemini API adapter: Gemini API users add both :inference and :gemini_ex; core/mock users add only :inference. Gemini CLI is retired. Antigravity is the current Google coding-agent SDK and is reached through the explicitly admitted ASM agent-session boundary, not the Gemini API adapter.

Jido governed execution is owned by jido_integration. The Jido-owned adapter implements Inference.Adapter from that repository and translates shared requests into governed control-plane execution.

That governed lane is required release scope for platforms that use universal auth authority. It remains outside the core :inference package so direct standalone adapters stay reusable, while governed deployments carry authority refs, credential handles or leases, target grants, and redacted trace evidence through the Jido-owned adapter.

Usage

client =
  Inference.Client.new!(
    adapter: Inference.Adapters.Mock,
    provider: :mock,
    model: "mock-fast",
    adapter_opts: [response_text: "hello"]
  )

{:ok, response} = Inference.complete(client, "Say hello")
Inference.Response.text(response)

Requests can also be built explicitly:

{:ok, request} =
  Inference.Request.from_messages([
    %{role: :system, content: "Be concise."},
    %{role: :user, content: "Summarize the result."}
  ])

{:ok, response} = Inference.complete(client, request)

Design Rules

The default test path is deterministic and mock-only.
Live provider calls are examples, not test requirements.
Provider dependencies are installed by the consuming application.
Adapter modules translate to and from provider libraries; they do not hide provider setup, credentials, or runtime requirements.
Clients admit model and local-model endpoints by default. Agent-session adapters require admitted_kinds: [:agent_session] so a generic inference caller cannot silently flatten a stateful coding-agent session.
Inference.Adapters.ASM is common-only. It validates options through ASM strict preflight, rejects provider-native tool/configuration keys, and does not expose ASM host tools until ASM has a proven all-provider tool contract. Custom ASM modules must provide an explicit :asm_options_module; only the default ASM module uses the built-in ASM.Options default.
Jido governed execution is owned by jido_integration, which implements Inference.Adapter from the Jido side.
Direct :inference adapters are standalone mechanics. They do not decide durable provider credential authority, target attachment, or workflow admission for governed execution.
Shared governed clients may carry explicit authority refs, endpoint refs, provider-account refs, credential refs, service-identity refs, target refs, and redaction values. The shared package rejects direct provider keys, endpoint auth, model-account secrets, service identity secrets, raw env functions, and adapter defaults beside that authority packet; Jido still owns the governed adapter and durable control-plane behavior.

Guides

Next Page → Architecture