View Source Agens.Serving (agens v0.1.1)
The Serving module provides functions for starting, stopping and running Servings.
Agens.Serving
accepts a GenServer
module or Nx.Serving
struct for processing messages.
Agens.Serving
is decoupled from Agens.Agent
in order to reuse a single LM across multiple agents. In most cases, however, you will only need to start one text generation serving to be used by most, if not all, agents.
In some cases, you may have additional servings for more specific use cases such as image generation, speech recognition, etc.
Servings were built with the Bumblebee
library in mind, as well as Nx.Serving
. GenServer
is supported for working with LM APIs instead, which may be more cost effective and easier to get started with.
Summary
Functions
Executes an Agens.Message
against an Agens.Serving
Starts an Agens.Serving
process
Stops an Agens.Serving
process
Functions
@spec run(Agens.Message.t()) :: String.t() | {:error, :serving_not_running}
Executes an Agens.Message
against an Agens.Serving
@spec start(Agens.Serving.Config.t()) :: {:ok, pid()}
Starts an Agens.Serving
process
@spec stop(atom()) :: :ok | {:error, :serving_not_found}
Stops an Agens.Serving
process