ExPmtiles.Cache (ExPmtiles v0.1.2)

View Source

A GenServer implementation of a multi-level cache for PMTiles file data.

This GenServer provides efficient access to PMTiles map data through several caching mechanisms:

Cache Levels

1. Directory Cache

  • Caches deserialized directory structures from the PMTiles file
  • Pre-populates cache for zoom levels 0-4 on startup
  • Persists cached directories to disk for faster restarts
  • Uses zlib compression for efficient storage
  • Shared across all requests to prevent duplicate S3 fetches

2. Tile Cache

  • Uses ETS tables to cache individual tile data
  • Implements LRU (Least Recently Used) eviction
  • Configurable maximum entries (default: 100,000)
  • Maintains hit/miss statistics
  • Automatic cleanup of idle entries (1 hour timeout)

3. Concurrent Request Handling

  • Prevents duplicate requests for the same tile
  • Coordinates multiple requests through a pending requests table
  • Broadcasts results to all waiting processes
  • 30-second timeout for waiting processes

Cache Files

Directory cache files are stored in priv/pmtiles_cache with the format: {bucket}_{filename}_directories.cache

Configuration

  • :max_entries - Maximum number of tiles to cache (default: 100,000)
  • :pmtiles_module - Module to use for PMTiles operations (configurable for testing)

Usage

# Start the cache for an S3 PMTiles file
{:ok, pid} = ExPmtiles.Cache.start_link(bucket: "maps", path: "map.pmtiles")

# Start the cache for a local PMTiles file
{:ok, pid} = ExPmtiles.Cache.start_link(bucket: nil, path: "/path/to/map.pmtiles")

# Get a tile by coordinates
case ExPmtiles.Cache.get_tile(pid, 10, 512, 256) do
  {:ok, tile_data} ->
    # Handle tile data
    tile_data
  {:error, reason} ->
    # Handle error
    nil
end

# Get cache statistics
stats = ExPmtiles.Cache.get_stats(pid)
# Returns: %{hits: 150, misses: 25}

Implementation Details

The GenServer maintains several ETS tables:

  • Main table: Caches tile data with timestamps for LRU eviction
  • Pending table: Tracks in-progress requests to prevent duplicates
  • Stats table: Maintains hit/miss statistics

The directory cache is maintained in the GenServer state and is progressively updated as new directories are accessed. This prevents duplicate S3 requests and directory deserialization operations.

Performance Features

  • Background population: Pre-loads directories for zoom levels 0-4
  • Persistent cache: Saves directories to disk for faster restarts
  • Compression: Uses zlib compression for cache files
  • Concurrent access: Multiple processes can safely access the same cache
  • Automatic cleanup: Removes idle entries every 5 minutes

Summary

Functions

Returns a specification to start this module under a supervisor.

Retrieves cache statistics.

Retrieves a tile from the cache by coordinates.

Starts a new PMTiles cache GenServer.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

get_stats(pid)

Retrieves cache statistics.

Returns hit and miss counts for the cache, useful for monitoring cache performance and effectiveness.

Parameters

  • pid - The cache process PID

Returns

  • %{hits: integer(), misses: integer()} - Cache statistics

Examples

iex> {:ok, pid} = ExPmtiles.Cache.start_link(bucket: "maps", path: "world.pmtiles")
iex> ExPmtiles.Cache.get_stats(pid)
%{hits: 150, misses: 25}

iex> # After some cache operations
iex> ExPmtiles.Cache.get_stats(pid)
%{hits: 175, misses: 30}

get_tile(pid, z, x, y)

Retrieves a tile from the cache by coordinates.

This function handles both cache hits and misses. If the tile is not in the cache, it will be fetched from the PMTiles file and cached for future requests.

Parameters

  • {bucket, path} - Tuple identifying the PMTiles file or pid identifying the cache process
  • z - Zoom level (integer)
  • x - X coordinate (integer)
  • y - Y coordinate (integer)

Returns

  • {:ok, tile_data} - Tile data as binary
  • {:error, reason} - Error retrieving tile (e.g., :tile_not_found, :timeout)
  • nil - If the cache process is not available

Examples

iex> ExPmtiles.Cache.get_tile({"maps", "world.pmtiles"}, 10, 512, 256)
{:ok, <<...>>}

iex> ExPmtiles.Cache.get_tile({"maps", "world.pmtiles"}, 25, 0, 0)
{:error, :tile_not_found}

iex> ExPmtiles.Cache.get_tile(pid, 10, 512, 256)
{:ok, <<...>>}

start_link(opts \\ [])

Starts a new PMTiles cache GenServer.

Creates a named GenServer process that manages caching for a specific PMTiles file. The process name is derived from the bucket and path to ensure uniqueness.

Parameters

  • opts - Keyword list of options:
    • :bucket - S3 bucket name (or nil for local files)
    • :path - Path to the PMTiles file
    • :max_entries - Maximum number of tiles to cache (default: 100,000)

Returns

  • {:ok, pid} - Successfully started cache process
  • {:error, reason} - Failed to start cache

Examples

iex> ExPmtiles.Cache.start_link(region: nil, bucket: "maps", path: "world.pmtiles")
{:ok, #PID<0.123.0>}

iex> ExPmtiles.Cache.start_link(bucket: nil, path: "/data/local.pmtiles")
{:ok, #PID<0.124.0>}

iex> ExPmtiles.Cache.start_link(bucket: "maps", path: "world.pmtiles", max_entries: 50_000)
{:ok, #PID<0.125.0>}