ExLLM.Providers.Shared.VisionFormatter behaviour (ex_llm v0.8.1)
View SourceUnified vision/multimodal content formatting for LLM providers.
This module standardizes the handling of images and other multimodal content across different providers, reducing code duplication.
Features:
- Image format validation and detection
- Base64 encoding/decoding
- Provider-specific formatting
- URL and file handling
- Content type detection
Summary
Callbacks
Callback for provider-specific vision content formatting.
Callback to check if a model supports vision.
Functions
Create a base64 image content part.
Detect media type from file extension or magic bytes.
Format messages containing vision content for a specific provider.
Check if messages contain vision content.
Create an image URL content part.
Load an image from a file path and encode it.
Check if a single message has vision content.
Validate that an image format is supported.
Callbacks
Callback for provider-specific vision content formatting.
Callback to check if a model supports vision.
Functions
Create a base64 image content part.
Detect media type from file extension or magic bytes.
Format messages containing vision content for a specific provider.
Examples
messages = [
%{
role: "user",
content: [
%{type: "text", text: "What's in this image?"},
%{type: "image_url", image_url: %{url: "https://example.com/cat.jpg"}}
]
}
]
formatted = VisionFormatter.format_messages(messages, :anthropic)
Check if messages contain vision content.
Create an image URL content part.
Load an image from a file path and encode it.
Returns a content part ready for inclusion in a message.
Options
:format
- Force output format (:base64 or :url):max_size
- Maximum file size in bytes:detail
- Image detail level ("low", "high", "auto")
Check if a single message has vision content.
@spec validate_media_type(String.t()) :: :ok | {:error, :unsupported_format}
Validate that an image format is supported.