ExLLM.Gemini.Tokens (ex_llm v0.5.0)

View Source

Google Gemini Token Counting API implementation.

Provides functionality to count tokens for content and generate content requests before sending them to the model. This helps estimate costs and ensure requests fit within model limits.

Summary

Functions

Counts tokens for the given content or generate content request.

Normalizes model name to include proper prefix.

Converts a CountTokensRequest to JSON format for the API.

Validates that the request has either contents or generate_content_request, but not both.

Types

options()

@type options() :: [{:config_provider, pid() | atom()}]

Functions

count_tokens(model, request, opts \\ [])

Counts tokens for the given content or generate content request.

Parameters

  • model - The model name (e.g., "gemini-2.0-flash")
  • request - CountTokensRequest with either contents or generate_content_request
  • opts - Options including :config_provider

Examples

# Count tokens for simple content
request = %CountTokensRequest{
  contents: [
    %Content{
      role: "user",
      parts: [%Part{text: "Hello world"}]
    }
  ]
}
{:ok, response} = ExLLM.Gemini.Tokens.count_tokens("gemini-2.0-flash", request)

# Count tokens for full generate request
request = %CountTokensRequest{
  generate_content_request: %GenerateContentRequest{
    contents: [...],
    system_instruction: %Content{...}
  }
}
{:ok, response} = ExLLM.Gemini.Tokens.count_tokens("gemini-2.0-flash", request)

normalize_model_name(name)

@spec normalize_model_name(String.t() | nil) :: {:ok, String.t()} | {:error, map()}

Normalizes model name to include proper prefix.

to_json(request)

Converts a CountTokensRequest to JSON format for the API.

validate_request(count_tokens_request)

@spec validate_request(ExLLM.Gemini.Tokens.CountTokensRequest.t()) ::
  :ok | {:error, map()}

Validates that the request has either contents or generate_content_request, but not both.