GenAI.Provider.LiteLLM.Media (Noizu Labs, GenAI Wrapper v0.3.2)

Copy Markdown

LiteLLM media-generation provider (ADR-016) — the proxy can host many media model types, so this declares the full set and dispatches by output modality:

  • :image -> POST /v1/images/generations (b64_json) [concrete]
  • :speech (TTS) -> POST /v1/audio/speech (raw audio bytes) [concrete]
  • :text (STT) -> POST /v1/audio/transcriptions (multipart) [concrete]
  • :music/:sfx/:video -> a deployment-configured passthrough endpoint [config]

image/speech/transcription reuse the shared OpenAICompat calls. music/sfx/video have no universal OpenAI-compatible endpoint, so they route to a per-modality path from config :genai, :litellm, media_endpoints: %{music: "/v1/...", video: "/v1/..."} (or req.settings[:endpoint]); response is decoded as JSON-b64/url or raw bytes. Unconfigured -> {:error, {:modality_not_configured, modality}}.

base_url + Bearer key come from config :genai, :litellm (or LITELLM_API_KEY).

Summary

Functions

chat(messages, tools, settings)
chat(model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)

Callback implementation for GenAI.InferenceProviderBehaviour.chat/7.

config_key()

Return config_key inference provide application config stored under :genai entry

default_encoder()
effective_settings(model, session, context, options \\ nil)

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

endpoint(model, settings, session, context, options \\ nil)

Prepare endpoint and method to make inference call to

headers(options)
headers(model, settings, session, context, options \\ nil)

Prepare request headers

request_body(model, messages, tools, settings, session, context, options \\ nil)

Prepare request body to be passed to inference call.

run(session, context, options \\ nil)

Build and run inference thread

standardize_model(model)

Callback implementation for GenAI.InferenceProviderBehaviour.standardize_model/1.

stream(session, context, options \\ nil)

Build and run inference thread in streaming mode

Functions

chat(messages, tools, settings)

chat(model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)

Callback implementation for GenAI.InferenceProviderBehaviour.chat/7.

config_key()

Return config_key inference provide application config stored under :genai entry

default_encoder()

effective_settings(model, session, context, options \\ nil)

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

endpoint(model, settings, session, context, options \\ nil)

Prepare endpoint and method to make inference call to

headers(options)

headers(model, settings, session, context, options \\ nil)

Prepare request headers

request_body(model, messages, tools, settings, session, context, options \\ nil)

Prepare request body to be passed to inference call.

run(session, context, options \\ nil)

Build and run inference thread

standardize_model(model)

Callback implementation for GenAI.InferenceProviderBehaviour.standardize_model/1.

stream(session, context, options \\ nil)

Build and run inference thread in streaming mode