Omni.Source behaviour (Omni v1.6.1)

Copy Markdown View Source

Behaviour for pluggable model data sources.

Experimental

This behaviour is experimental and may change in a future release.

A source answers one question: give me the models for this provider. Omni.Provider.load_models/2 resolves the configured source and calls fetch/2; where the data comes from — a bundled snapshot, another package's catalog, a file, a network fetch — is entirely the source's concern.

Omni ships Omni.Sources.ModelsDev (the default, reading a bundled models.dev snapshot). See Omni.Provider for how sources are configured and resolved.

The fetch/2 contract

fetch/2 receives the provider module and a keyword list of options. Omni.Provider.load_models/2 builds the options in layers: the module's canonical id as provider_id:, then the source's configured opts (from {module, opts} config), then the remaining call-site opts — later layers winning. All options are passed through, so sources may define their own.

Every source must honor :provider_id — the provider's canonical Omni id, a snake_cased models.dev catalog key (:anthropic, :fireworks_ai). Translating it to the source's native catalog key is the source's job: Omni.Sources.ModelsDev hyphenates it, Omni.Sources.LLMDB uses it verbatim.

On success, return {:ok, models} — an empty list is a valid result (a known provider whose models were all filtered out). Return {:error, :unknown_provider} when the provider is not in the source's catalog; any other error term is also legal and treated the same way by load_models/2 (a logged warning and an empty model list — there is no cross-source fallback).

Model struct invariants

Sources return ready-to-use %Omni.Model{} structs, not intermediate maps. Every returned model must satisfy:

  • id — present; the provider's model identifier string
  • provider — the module passed to fetch/2
  • dialect — a resolved dialect module, never nil

Beware the dialect invariant: Omni.Model enforces the :dialect key, but @enforce_keys only rejects an omitted key — Omni.Model.new(dialect: nil) builds silently. Dialect resolution is the source's responsibility; a model whose dialect cannot be resolved must be skipped (with a warning) or raise, per the source's error policy.

Use Omni.Provider.build_model/2 to construct models from string-keyed data maps — it implements the dialect precedence (data "dialect" wins, the provider's declared dialect is the fallback), date parsing, and field defaults shared by the built-in sources.

Caching expensive work

fetch/2 is called once per provider within a load pass (a dozen times for the built-ins at boot). Sources that derive models from a shared expensive artifact — a large decoded snapshot, a network response — should wrap that work in memo/2: the result is computed once per pass and garbage collected when the pass ends, rather than lingering for the VM's lifetime. See Omni.Sources.ModelsDev for an example.

Summary

Types

t()

A source in configuration: a module or a {module, opts} tuple.

Callbacks

fetch(provider, opts)

Returns the models for the given provider module.

Functions

memo(key, fun)

Memoizes fun's result under key for the duration of the enclosing cache scope (see with_cache/1).

with_cache(fun)

Runs fun inside a cache scope for memo/2, on the calling process.

Types

t()

@type t() :: module() | {module(), keyword()}

A source in configuration: a module or a {module, opts} tuple.

Callbacks

fetch(provider, opts)

@callback fetch(provider :: module(), opts :: keyword()) ::
  {:ok, [Omni.Model.t()]} | {:error, term()}

Returns the models for the given provider module.

See the module documentation for the option-merging semantics, the {:error, :unknown_provider} convention, and the invariants returned models must satisfy.

Functions

memo(key, fun)

@spec memo(term(), (-> result)) :: result when result: var

Memoizes fun's result under key for the duration of the enclosing cache scope (see with_cache/1).

Outside a scope — a one-off Omni.Provider.load_models/2 call, for example — there is nothing to share the result with, so fun simply runs. Keys are namespaced by convention: use {YourSource, :name} tuples.

with_cache(fun)

@spec with_cache((-> result)) :: result when result: var

Runs fun inside a cache scope for memo/2, on the calling process.

Omni.Provider.load/1 wraps each load pass in a scope so that every memo/2 result is shared across the pass's fetch/2 calls and released when the pass ends — the scope is cleared and the process's garbage is collected, so pass-transient data (like a decoded catalog) does not linger on the caller's heap. Nested calls reuse the enclosing scope.