GenAI.InferenceProviderBehaviour behaviour (GenAI Core v0.2.0)

Link to this section Summary

Callbacks

Return config_key inference provide application config stored under :genai entry

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

Prepare endpoint and method to make inference call to

Prepare request body to be passed to inference call.

Build and run inference thread

Link to this section Types

Link to this type

completion()

@type completion() :: any()
@type context() :: any()
@type headers() :: list()
@type messages() :: list()
@type method() :: :get | :post | :put | :delete | :option | :patch
@type model() :: any()
@type options() :: any()
Link to this type

request_body()

@type request_body() :: any()
@type session() :: any()
@type settings() :: map()
@type tools() :: list() | nil
@type uri() :: url()
@type url() :: String.t()

Link to this section Callbacks

Link to this callback

chat(any, any, any, any, any, any, any)

@callback chat(any(), any(), any(), any(), any(), any(), any()) ::
  {:ok, term()} | {:error, term()}
@callback config_key() :: atom()

Return config_key inference provide application config stored under :genai entry

Link to this callback

effective_settings(model, session, context, options)

@callback effective_settings(model(), session(), context(), options()) ::
  {:ok, {settings(), session()}} | {:error, term()}

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

Link to this callback

endpoint(model, settings, session, context, options)

@callback endpoint(model(), settings(), session(), context(), options()) ::
  {:ok, {method(), uri()}}
  | {:ok, {{method(), uri()}, session()}}
  | {:error, term()}

Prepare endpoint and method to make inference call to

Link to this callback

headers(model, settings, session, context, options)

@callback headers(model(), settings(), session(), context(), options()) ::
  {:ok, headers()} | {:ok, {headers(), session()}} | {:error, term()}

Prepare request headers

Link to this callback

request_body(model, messages, tools, settings, session, context, options)

@callback request_body(
  model(),
  messages(),
  tools(),
  settings(),
  session(),
  context(),
  options()
) ::
  {:ok, headers()} | {:ok, {headers(), session()}} | {:error, term()}

Prepare request body to be passed to inference call.

Link to this callback

run(session, context, options)

@callback run(session(), context(), options()) ::
  {:ok, {completion(), session()}} | {:error, term()}

Build and run inference thread

Link to this callback

standardize_model(model)

@callback standardize_model(model()) :: model()