GenAI.InferenceProvider.DefaultProvider (GenAI Core v0.2.0)
Inference Provider Default Provider.
Link to this section Summary
Functions
Sends a chat completion request to the Mistral API.
This function constructs the request body based on the provided messages, tools, and settings, sends the request to the Mistral API, and returns a GenAI.ChatCompletion
struct with the response.
Low level inference, pass in model, messages, tools, and various settings to prepare final provider specific API requires.
Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.
Prepare endpoint and method to make inference call to
Prepare request headers
Prepare request body to be passed to inference call.
Run inference and return completion response and updated session
Link to this section Functions
chat(module, messages, tools, settings)
Sends a chat completion request to the Mistral API.
This function constructs the request body based on the provided messages, tools, and settings, sends the request to the Mistral API, and returns a GenAI.ChatCompletion
struct with the response.
chat(module, model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)
Low level inference, pass in model, messages, tools, and various settings to prepare final provider specific API requires.
do_chat(module, messages, tools, settings)
do_chat(module, model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)
do_effective_settings(module, model, session, context, options)
do_endpoint(_, model, settings, session, context, options)
do_headers(module, model, settings, session, context, options)
do_request_body(module, model, messages, tools, settings, session, context, options)
do_run(module, session, context, options \\ nil)
effective_settings(module, model, session, context, options \\ nil)
Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.
endpoint(module, model, settings, session, context, options \\ nil)
Prepare endpoint and method to make inference call to
headers(module, options)
headers(module, model, settings, session, context, options \\ nil)
Prepare request headers
request_body(module, model, messages, tools, settings, session, context, options \\ nil)
Prepare request body to be passed to inference call.
run(module, session, context, options \\ nil)
Run inference and return completion response and updated session