View Source Agens.Serving.Config (agens v0.2.0)

The Config struct represents the configuration for a Serving process.

Fields

  • :name - The unique name for the Serving process.
  • :serving - The module implementing the Agens.Serving behaviour (i.e. one that calls use Agens.Serving).
  • :prefixes - An Agens.Prefixes struct of custom prompt prefixes. If nil, the default prefixes returned by Agens.Prefixes.default/0 will be used.
  • :args - Additional arguments passed through to the Serving module on start. Available to the Serving's Agens.Serving.start/1 callback via the initial state.config.args and typically used to configure the backend (model name, API base URL, credentials, etc).
  • :timeout - Timeout in milliseconds for LM inference. Defaults to 60_000.

Summary

Types

t()

Types

Link to this type

t()

View Source 
@type t() :: %Agens.Serving.Config{
  args: keyword(),
  name: atom(),
  prefixes: Agens.Prefixes.t() | nil,
  serving: module(),
  timeout: non_neg_integer()
}