GenAI.Provider.LocalLLama (Noizu Labs, GenAI Wrapper v0.0.3)

This module implements the GenAI provider for Local AI.

Link to this section Summary

Functions

Sends a chat completion request to the LocalLlama

Retrieves a list of available Local models.

Link to this section Functions

Link to this function

chat(messages, tools, settings)

Sends a chat completion request to the LocalLlama

This function constructs the request body based on the provided messages, tools, and settings, sends the request to the ExLLama instanceI, and returns a GenAI.ChatCompletion struct with the response.

Link to this function

models(settings \\ [])

Retrieves a list of available Local models.

This function calls the Local API to retrieve a list of models and returns them as a list of GenAI.Model structs.