ReqLLM.Providers.GoogleVertex.TokenCache (ReqLLM v1.14.0)

View Source

OAuth2 token cache for Google Vertex AI.

Caches access tokens per service account to avoid expensive token generation on every request.

Lifecycle

  • Started by ReqLLM.Application supervision tree
  • One cache per node (not distributed)
  • Tokens cached for 55 minutes (5 minute safety margin)

Usage

# Provider calls this instead of Auth.get_access_token/1 directly
{:ok, token} = TokenCache.get_or_refresh(service_account_json_path)

Cache Key

For file paths: the path string is used as the cache key. For JSON strings or maps: the client_email field is used as the cache key.

This allows multiple service accounts to be used simultaneously with independent token caches.

Expiry & Refresh

Tokens are cached for 55 minutes (5 minute safety margin before 1 hour expiry). The GenServer serializes concurrent refresh requests to prevent duplicate token fetches when the cache is empty or expired.

Summary

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

clear_all()

Clears all cached tokens.

get_or_refresh(service_account)

Retrieves a cached token or fetches a fresh one if expired.

invalidate(cache_key)

Invalidates cached token for a service account.

start_link(opts)

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

clear_all()

@spec clear_all() :: :ok

Clears all cached tokens.

Useful for testing.

get_or_refresh(service_account)

@spec get_or_refresh(service_account :: String.t() | map()) ::
  {:ok, access_token :: String.t()} | {:error, term()}

Retrieves a cached token or fetches a fresh one if expired.

This is the only function providers should call. It handles:

  • Cache hits (fast path)
  • Cache misses (slow path with fetch)
  • Expiry checking
  • Concurrent request deduplication

Accepts credentials in multiple formats:

  • File path (string, if file exists) - uses path as cache key
  • JSON string (string, if not a file) - uses client_email as cache key
  • Map (already parsed) - uses client_email as cache key

Examples

iex> TokenCache.get_or_refresh("/path/to/service-account.json")
{:ok, "ya29.c.Kl6iB..."}

iex> TokenCache.get_or_refresh(~s({"client_email": "...", "private_key": "..."}))
{:ok, "ya29.c.Kl6iB..."}

iex> TokenCache.get_or_refresh("/invalid/path.json")
{:error, :enoent}

invalidate(cache_key)

@spec invalidate(cache_key :: String.t()) :: :ok

Invalidates cached token for a service account.

Useful for testing or when credentials are rotated.

The cache_key should match what was used for caching:

  • File path if credentials were provided as a file path
  • client_email if credentials were provided as JSON string or map

start_link(opts)