View Source TextGenerationInference.TextGenerationInference (text_generation_inference v0.1.2)

Provides API endpoints related to text generation inference

Link to this section Summary

Functions

Generate tokens if stream == false or a stream of token if stream == true

Generate tokens

Generate a stream of token using Server-Sent Events

Text Generation Inference endpoint info

Prometheus metrics scrape endpoint

Link to this section Functions

Link to this function

compat_generate(body, opts \\ [])

View Source

Generate tokens if stream == false or a stream of token if stream == true

Link to this function

generate(body, opts \\ [])

View Source

Generate tokens

Link to this function

generate_stream(body, opts \\ [])

View Source
@spec generate_stream(
  TextGenerationInference.GenerateRequest.t(),
  keyword()
) :: {:ok, binary()} | {:error, binary()}

Generate a stream of token using Server-Sent Events

Link to this function

get_model_info(opts \\ [])

View Source
@spec get_model_info(keyword()) :: {:ok, TextGenerationInference.Info.t()} | :error

Text Generation Inference endpoint info

@spec metrics(keyword()) :: {:ok, binary()} | :error

Prometheus metrics scrape endpoint