GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpec (google_api_ai_platform v0.27.0)
View SourceConfiguration for Speculative Decoding.
Attributes
-
draftModelSpeculation
(type:GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecDraftModelSpeculation.t
, default:nil
) - draft model speculation. -
ngramSpeculation
(type:GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation.t
, default:nil
) - N-Gram speculation. -
speculativeTokenCount
(type:integer()
, default:nil
) - The number of speculative tokens to generate at each step.
Summary
Functions
Unwrap a decoded JSON object into its complex fields.
Types
@type t() :: %GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpec{ draftModelSpeculation: GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecDraftModelSpeculation.t() | nil, ngramSpeculation: GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation.t() | nil, speculativeTokenCount: integer() | nil }