View Source Agens.Serving.Config (agens v0.1.3)

The Config struct represents the configuration for a Serving process.

Fields

  • :name - The unique name for the Serving process.
  • :serving - The Nx.Serving struct or GenServer module for the Agens.Serving.
  • :prefixes - An Agens.Prefixes struct of custom prompt prefixes. If nil, default prompt prefixes will be used instead. Default prompt prefixes can also be overridden by using the prefixes options in Agens.Supervisor.
  • :finalize - A function that accepts the prepared prompt (including any applied prefixes) and returns a modified version of the prompt. Useful for wrapping the prompt or applying final processing before sending to the LM for inference. If nil, the prepared prompt will be used as-is.
  • :args - Additional arguments to be passed to the Nx.Serving or GenServer module. See the Nx.Serving or GenServer documentation for more information.

Summary

Types

@type t() :: %Agens.Serving.Config{
  args: keyword(),
  finalize: (String.t() -> String.t()) | nil,
  name: atom(),
  prefixes: Agens.Prefixes.t() | nil,
  serving: Nx.Serving.t() | module()
}