GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation (google_api_ai_platform v0.27.0)
View SourceN-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.
Attributes
-
ngramSize
(type:integer()
, default:nil
) - The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.
Summary
Functions
Unwrap a decoded JSON object into its complex fields.
Types
@type t() :: %GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation{ ngramSize: integer() | nil }