ExLLM.Providers.Shared.ModelFetcher behaviour (ex_llm v0.8.1)
View SourceUnified behavior and utilities for fetching models from LLM provider APIs.
This module standardizes the pattern of:
- Fetching models from provider APIs
- Filtering out non-LLM models
- Parsing model metadata
- Transforming to common format
- Integrating with ModelLoader for caching
Usage
Adapters can implement the behavior callbacks or use the helper functions to reduce code duplication.
Summary
Callbacks
Callback to fetch models from the provider's API.
Callback to filter out non-LLM models.
Callback to parse a raw API model into Types.Model struct.
Callback to transform config-based model data.
Functions
Fetch models from a standard OpenAI-compatible endpoint.
Standard implementation for list_models that integrates with ModelLoader.
Parse a model from API response to Types.Model struct.
Helper to process raw models through filter and parse pipeline.
Standard model filtering for common patterns.
Transform a model from YAML config to Types.Model struct.
Callbacks
Callback to fetch models from the provider's API.
Should return {:ok, raw_models} or {:error, reason}.
Callback to filter out non-LLM models.
Some providers return embeddings, whisper, etc. that we want to exclude.
@callback parse_api_model(model :: map()) :: ExLLM.Types.Model.t()
Callback to parse a raw API model into Types.Model struct.
@callback transform_model_config(model_id :: atom() | String.t(), config :: map()) :: ExLLM.Types.Model.t()
Callback to transform config-based model data.
Used when loading from YAML configuration.
Functions
@spec fetch_openai_compatible_models(String.t(), String.t(), keyword()) :: {:ok, [map()]} | {:error, term()}
Fetch models from a standard OpenAI-compatible endpoint.
Many providers use the same /v1/models endpoint format.
@spec list_models_with_loader(module(), atom(), keyword()) :: {:ok, [ExLLM.Types.Model.t()]} | {:error, term()}
Standard implementation for list_models that integrates with ModelLoader.
This is the main entry point that adapters should use.
Example
def list_models(options \ []) do
ModelFetcher.list_models_with_loader(__MODULE__, :myprovider, options)
end
@spec parse_standard_model(map(), atom(), keyword()) :: ExLLM.Types.Model.t()
Parse a model from API response to Types.Model struct.
Provides sensible defaults and uses ModelUtils for formatting.
@spec process_models([map()], module(), atom(), keyword()) :: [ExLLM.Types.Model.t()]
Helper to process raw models through filter and parse pipeline.
Standard model filtering for common patterns.
Filters out:
- Embedding models
- Whisper/audio models
- Image generation models
- Old deprecated models
@spec transform_standard_config(atom() | String.t(), map(), atom()) :: ExLLM.Types.Model.t()
Transform a model from YAML config to Types.Model struct.
Standard implementation that most adapters can use.