View Source TextGenerationInference.Info (text_generation_inference v0.1.2)
Provides struct and type for Info
Link to this section Summary
Link to this section Types
@type t() :: %TextGenerationInference.Info{ docker_label: String.t() | nil, max_batch_total_tokens: integer(), max_best_of: integer(), max_concurrent_requests: integer(), max_input_length: integer(), max_stop_sequences: integer(), max_total_tokens: integer(), max_waiting_tokens: integer(), model_device_type: String.t(), model_dtype: String.t(), model_id: String.t(), model_pipeline_tag: String.t() | nil, model_sha: String.t() | nil, sha: String.t() | nil, validation_workers: integer(), version: String.t(), waiting_served_ratio: number() }