GenAI.InferenceProvider.DefaultProvider
(GenAI Core v0.3.1)
Copy Markdown
Inference Provider Default Provider.
Summary
Functions
Sends a chat completion request to the Mistral API.
This function constructs the request body based on the provided messages, tools, and settings, sends the request to the Mistral API, and returns a GenAI.ChatCompletion struct with the response.
Low level inference, pass in model, messages, tools, and various settings to prepare final provider specific API requires.
Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.
Prepare endpoint and method to make inference call to
Prepare request headers
Prepare request body to be passed to inference call.
Run inference and return completion response and updated session
Run inference in streaming mode and return completion response and updated session
Functions
Sends a chat completion request to the Mistral API.
This function constructs the request body based on the provided messages, tools, and settings, sends the request to the Mistral API, and returns a GenAI.ChatCompletion struct with the response.
chat(module, model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)
Low level inference, pass in model, messages, tools, and various settings to prepare final provider specific API requires.
do_chat(module, model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)
Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.
Prepare endpoint and method to make inference call to
Prepare request headers
Prepare request body to be passed to inference call.
Run inference and return completion response and updated session
Run inference in streaming mode and return completion response and updated session