GenAI.InferenceProviderBehaviour behaviour (GenAI Core v0.3.1)

Copy Markdown

Summary

Types

completion()
context()
headers()
messages()
method()
modality()
model()
options()
request_body()
session()
settings()
tools()
uri()
url()

Callbacks

chat(any, any, any, any, any, any, any)
config_key()

Return config_key inference provide application config stored under :genai entry

effective_settings(model, session, context, options)

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

endpoint(model, settings, session, context, options)

Prepare endpoint and method to make inference call to

generate_media(request, options)

Generate non-text media (text/parts -> image/audio/video). Sync providers return the bytes; long-running (audio/video) providers return a Job to poll. Default: unsupported.

headers(model, settings, session, context, options)

Prepare request headers

request_body(model, messages, tools, settings, session, context, options)

Prepare request body to be passed to inference call.

run(session, context, options)

Build and run inference thread

standardize_model(model)
stream(session, context, options)

Build and run inference thread in streaming mode

supported_modalities()

The (input, output, mode) modality triples this provider supports, e.g. [%{input: [:text], output: :image, mode: :sync}]. genai-core owns the modality vocab; each provider declares its own list. Default (text-only chat) is provided.

Types

completion()

@type completion() :: any()

context()

@type context() :: any()

headers()

@type headers() :: list()

messages()

@type messages() :: list()

method()

@type method() :: :get | :post | :put | :delete | :option | :patch

modality()

@type modality() :: :text | :image | :speech | :music | :sfx | :video | :document

model()

@type model() :: any()

options()

@type options() :: any()

request_body()

@type request_body() :: any()

session()

@type session() :: any()

settings()

@type settings() :: map()

tools()

@type tools() :: list() | nil

uri()

@type uri() :: url()

url()

@type url() :: String.t()

Callbacks

chat(any, any, any, any, any, any, any)

@callback chat(any(), any(), any(), any(), any(), any(), any()) ::
  {:ok, term()} | {:error, term()}

config_key()

@callback config_key() :: atom()

Return config_key inference provide application config stored under :genai entry

effective_settings(model, session, context, options)

@callback effective_settings(model(), session(), context(), options()) ::
  {:ok, {settings(), session()}} | {:error, term()}

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

endpoint(model, settings, session, context, options)

@callback endpoint(model(), settings(), session(), context(), options()) ::
  {:ok, {method(), uri()}}
  | {:ok, {{method(), uri()}, session()}}
  | {:error, term()}

Prepare endpoint and method to make inference call to

generate_media(request, options)

@callback generate_media(request :: GenAI.Media.Request.t(), options :: keyword()) ::
  {:ok, %{data: binary(), mime: String.t(), meta: map()}}
  | {:ok, GenAI.Media.Job.t()}
  | {:error, term()}

Generate non-text media (text/parts -> image/audio/video). Sync providers return the bytes; long-running (audio/video) providers return a Job to poll. Default: unsupported.

headers(model, settings, session, context, options)

@callback headers(model(), settings(), session(), context(), options()) ::
  {:ok, headers()} | {:ok, {headers(), session()}} | {:error, term()}

Prepare request headers

request_body(model, messages, tools, settings, session, context, options)

@callback request_body(
  model(),
  messages(),
  tools(),
  settings(),
  session(),
  context(),
  options()
) ::
  {:ok, headers()} | {:ok, {headers(), session()}} | {:error, term()}

Prepare request body to be passed to inference call.

run(session, context, options)

@callback run(session(), context(), options()) ::
  {:ok, {completion(), session()}} | {:error, term()}

Build and run inference thread

standardize_model(model)

@callback standardize_model(model()) :: model()

stream(session, context, options)

@callback stream(session(), context(), options()) ::
  {:ok, {completion(), session()}} | {:error, term()} | :nyi

Build and run inference thread in streaming mode

supported_modalities()

@callback supported_modalities() :: [
  %{input: [modality()], output: modality(), mode: :sync | :async}
]

The (input, output, mode) modality triples this provider supports, e.g. [%{input: [:text], output: :image, mode: :sync}]. genai-core owns the modality vocab; each provider declares its own list. Default (text-only chat) is provided.