GenAI.Provider.LocalLLama (Noizu Labs, GenAI Wrapper v0.0.3)
This module implements the GenAI provider for Local AI.
Link to this section Summary
Functions
Sends a chat completion request to the LocalLlama
Retrieves a list of available Local models.
Link to this section Functions
Link to this function
chat(messages, tools, settings)
Sends a chat completion request to the LocalLlama
This function constructs the request body based on the provided messages, tools, and settings, sends the request to the ExLLama instanceI, and returns a GenAI.ChatCompletion
struct with the response.
Link to this function
models(settings \\ [])
Retrieves a list of available Local models.
This function calls the Local API to retrieve a list of models and returns them as a list of GenAI.Model
structs.