ExLLM.Gemini.Tokens (ex_llm v0.5.0)
View SourceGoogle Gemini Token Counting API implementation.
Provides functionality to count tokens for content and generate content requests before sending them to the model. This helps estimate costs and ensure requests fit within model limits.
Summary
Functions
Counts tokens for the given content or generate content request.
Normalizes model name to include proper prefix.
Converts a CountTokensRequest to JSON format for the API.
Validates that the request has either contents or generate_content_request, but not both.
Types
Functions
@spec count_tokens(String.t(), ExLLM.Gemini.Tokens.CountTokensRequest.t(), options()) :: {:ok, ExLLM.Gemini.Tokens.CountTokensResponse.t()} | {:error, term()}
Counts tokens for the given content or generate content request.
Parameters
model
- The model name (e.g., "gemini-2.0-flash")request
- CountTokensRequest with either contents or generate_content_requestopts
- Options including:config_provider
Examples
# Count tokens for simple content
request = %CountTokensRequest{
contents: [
%Content{
role: "user",
parts: [%Part{text: "Hello world"}]
}
]
}
{:ok, response} = ExLLM.Gemini.Tokens.count_tokens("gemini-2.0-flash", request)
# Count tokens for full generate request
request = %CountTokensRequest{
generate_content_request: %GenerateContentRequest{
contents: [...],
system_instruction: %Content{...}
}
}
{:ok, response} = ExLLM.Gemini.Tokens.count_tokens("gemini-2.0-flash", request)
Normalizes model name to include proper prefix.
@spec to_json(ExLLM.Gemini.Tokens.CountTokensRequest.t()) :: map()
Converts a CountTokensRequest to JSON format for the API.
@spec validate_request(ExLLM.Gemini.Tokens.CountTokensRequest.t()) :: :ok | {:error, map()}
Validates that the request has either contents or generate_content_request, but not both.