GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation (google_api_ai_platform v0.27.0)

View Source

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.

Attributes

  • ngramSize (type: integer(), default: nil) - The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.

Summary

Functions

Unwrap a decoded JSON object into its complex fields.

Types

t()

@type t() ::
  %GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation{
    ngramSize: integer() | nil
  }

Functions

decode(value, options)

@spec decode(struct(), keyword()) :: struct()

Unwrap a decoded JSON object into its complex fields.