View Source TextGenerationInference.TextGenerationInference (text_generation_inference v0.1.1)
Provides API endpoints related to text generation inference
Link to this section Summary
Functions
Generate tokens if stream == false
or a stream of token if stream == true
Generate tokens
Generate a stream of token using Server-Sent Events
Text Generation Inference endpoint info
Prometheus metrics scrape endpoint
Link to this section Functions
@spec compat_generate( TextGenerationInference.CompatGenerateRequest.t(), keyword() ) :: {:ok, TextGenerationInference.GenerateResponse.t()} | {:error, TextGenerationInference.ErrorResponse.t()}
Generate tokens if stream == false
or a stream of token if stream == true
@spec generate( TextGenerationInference.GenerateRequest.t(), keyword() ) :: {:ok, TextGenerationInference.GenerateResponse.t()} | {:error, TextGenerationInference.ErrorResponse.t()}
Generate tokens
@spec generate_stream( TextGenerationInference.GenerateRequest.t(), keyword() ) :: {:ok, binary()} | {:error, binary()}
Generate a stream of token using Server-Sent Events
@spec get_model_info(keyword()) :: {:ok, TextGenerationInference.Info.t()} | :error
Text Generation Inference endpoint info
Prometheus metrics scrape endpoint