ReqLLM.Providers.GoogleVertex.TokenCache (ReqLLM v1.17.0)

View Source

OAuth2 token cache for Google Vertex AI.

Caches access tokens per Google auth source to avoid expensive token generation on every request.

Lifecycle

  • Started by ReqLLM.Application supervision tree
  • One cache per node (not distributed)
  • Tokens cached until the auth source's reported expiry

Usage

# Provider calls this instead of Auth.get_access_token/1 directly
{:ok, token} = TokenCache.get_or_refresh(:adc)

Cache Key

For ADC: {:adc, scope} is used as the cache key, where scope identifies the credential location (env var path, inline JSON, well-known file, or metadata server) so swapping credentials at runtime does not serve stale tokens. For service account file paths: the path string is used as the cache key. For service account JSON strings or maps: the client_email field is used as the cache key.

This allows ADC and multiple service accounts to be used simultaneously with independent token caches.

Expiry & Refresh

Tokens are cached until the token expiry reported by the auth source. The GenServer serializes concurrent refresh requests to prevent duplicate token fetches when the cache is empty or expired.

Summary

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

clear_all()

Clears all cached tokens.

get_or_refresh(source, opts \\ [])

Retrieves a cached token or fetches a fresh one if expired.

invalidate(cache_key)

Invalidates cached token for an auth source.

start_link(opts)

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

clear_all()

@spec clear_all() :: :ok

Clears all cached tokens.

Useful for testing.

get_or_refresh(source, opts \\ [])

Retrieves a cached token or fetches a fresh one if expired.

This is the only function providers should call. It handles:

  • Cache hits (fast path)
  • Cache misses (slow path with fetch)
  • Expiry checking
  • Concurrent request deduplication

Accepts auth sources in multiple formats:

  • :adc - uses Application Default Credentials
  • {:service_account, credentials} - uses explicit service account credentials
  • Legacy service account credentials directly

Examples

iex> TokenCache.get_or_refresh(:adc)
{:ok, "ya29.c.Kl6iB..."}

iex> TokenCache.get_or_refresh({:service_account, "/path/to/service-account.json"})
{:ok, "ya29.c.Kl6iB..."}

iex> TokenCache.get_or_refresh("/invalid/path.json")
{:error, :enoent}

invalidate(cache_key)

@spec invalidate(cache_key :: term()) :: :ok

Invalidates cached token for an auth source.

Useful for testing or when credentials are rotated.

The cache key should match what was used for caching.

start_link(opts)