GenAI.InferenceProvider.DefaultProvider (GenAI Core v0.2.0)

Inference Provider Default Provider.

Link to this section Summary

Functions

Sends a chat completion request to the Mistral API. This function constructs the request body based on the provided messages, tools, and settings, sends the request to the Mistral API, and returns a GenAI.ChatCompletion struct with the response.

Low level inference, pass in model, messages, tools, and various settings to prepare final provider specific API requires.

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

Prepare endpoint and method to make inference call to

Run inference and return completion response and updated session

Link to this section Functions

Link to this function

chat(module, messages, tools, settings)

Sends a chat completion request to the Mistral API. This function constructs the request body based on the provided messages, tools, and settings, sends the request to the Mistral API, and returns a GenAI.ChatCompletion struct with the response.

Link to this function

chat(module, model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)

Low level inference, pass in model, messages, tools, and various settings to prepare final provider specific API requires.

Link to this function

do_chat(module, messages, tools, settings)

Link to this function

do_chat(module, model, messages, tools, hyper_parameters, provider_settings \\ [], context \\ nil, options \\ nil)

Link to this function

do_effective_settings(module, model, session, context, options)

Link to this function

do_endpoint(_, model, settings, session, context, options)

Link to this function

do_headers(module, model, settings, session, context, options)

Link to this function

do_request_body(module, model, messages, tools, settings, session, context, options)

Link to this function

do_run(module, session, context, options \\ nil)

Link to this function

effective_settings(module, model, session, context, options \\ nil)

Obtain map of effective settings: settings, model_settings, provider_settings, config_settings, etc.

Link to this function

endpoint(module, model, settings, session, context, options \\ nil)

Prepare endpoint and method to make inference call to

Link to this function

headers(module, options)

Link to this function

headers(module, model, settings, session, context, options \\ nil)

Prepare request headers

Link to this function

request_body(module, model, messages, tools, settings, session, context, options \\ nil)

Prepare request body to be passed to inference call.

Link to this function

run(module, session, context, options \\ nil)

Run inference and return completion response and updated session

Link to this function

standardize_model(model)

Link to this function

standardize_model(module, encoder, model)