GenAI.InferenceProviderBehaviour behaviour (GenAI Core v0.2.0)
Link to this section Summary
Callbacks
Return config_key inference provide application config stored under :genai entry
Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.
Prepare endpoint and method to make inference call to
Prepare request headers
Prepare request body to be passed to inference call.
Build and run inference thread
Link to this section Types
Link to this type
completion()
@type completion() :: any()
Link to this type
context()
@type context() :: any()
Link to this type
headers()
@type headers() :: list()
Link to this type
messages()
@type messages() :: list()
Link to this type
method()
@type method() :: :get | :post | :put | :delete | :option | :patch
Link to this type
model()
@type model() :: any()
Link to this type
options()
@type options() :: any()
Link to this type
request_body()
@type request_body() :: any()
Link to this type
session()
@type session() :: any()
Link to this type
settings()
@type settings() :: map()
Link to this type
tools()
@type tools() :: list() | nil
Link to this type
uri()
@type uri() :: url()
Link to this type
url()
@type url() :: String.t()
Link to this section Callbacks
Link to this callback
chat(any, any, any, any, any, any, any)
Link to this callback
config_key()
@callback config_key() :: atom()
Return config_key inference provide application config stored under :genai entry
Link to this callback
effective_settings(model, session, context, options)
@callback effective_settings(model(), session(), context(), options()) :: {:ok, {settings(), session()}} | {:error, term()}
Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.
Link to this callback
endpoint(model, settings, session, context, options)
@callback endpoint(model(), settings(), session(), context(), options()) :: {:ok, {method(), uri()}} | {:ok, {{method(), uri()}, session()}} | {:error, term()}
Prepare endpoint and method to make inference call to
Link to this callback
headers(model, settings, session, context, options)
@callback headers(model(), settings(), session(), context(), options()) :: {:ok, headers()} | {:ok, {headers(), session()}} | {:error, term()}
Prepare request headers
Link to this callback
request_body(model, messages, tools, settings, session, context, options)
@callback request_body( model(), messages(), tools(), settings(), session(), context(), options() ) :: {:ok, headers()} | {:ok, {headers(), session()}} | {:error, term()}
Prepare request body to be passed to inference call.
Link to this callback
run(session, context, options)
@callback run(session(), context(), options()) :: {:ok, {completion(), session()}} | {:error, term()}
Build and run inference thread
Link to this callback