View Source Agens.Serving.Config (agens v0.1.3)
The Config struct represents the configuration for a Serving process.
Fields
:name
- The unique name for the Serving process.:serving
- TheNx.Serving
struct orGenServer
module for theAgens.Serving
.:prefixes
- AnAgens.Prefixes
struct of custom prompt prefixes. Ifnil
, default prompt prefixes will be used instead. Default prompt prefixes can also be overridden by using theprefixes
options inAgens.Supervisor
.:finalize
- A function that accepts the prepared prompt (including any applied prefixes) and returns a modified version of the prompt. Useful for wrapping the prompt or applying final processing before sending to the LM for inference. Ifnil
, the prepared prompt will be used as-is.:args
- Additional arguments to be passed to theNx.Serving
orGenServer
module. See the Nx.Serving or GenServer documentation for more information.
Summary
Types
@type t() :: %Agens.Serving.Config{ args: keyword(), finalize: (String.t() -> String.t()) | nil, name: atom(), prefixes: Agens.Prefixes.t() | nil, serving: Nx.Serving.t() | module() }