Search documentation of snmpkit

# SnmpKit.SnmpLib.Cache (snmpkit v1.2.0)

Intelligent caching system for SNMP operations with adaptive strategies.

This module provides sophisticated caching capabilities designed to optimize SNMP polling performance in high-throughput environments. Based on patterns proven in the DDumb project for managing thousands of concurrent device polls.

## Features

- Multi-Level Caching: L1 (in-memory), L2 (ETS), L3 (persistent storage)
- Adaptive TTL: Dynamic cache expiration based on data volatility
- Smart Invalidation: Automatic cache invalidation based on data patterns
- Compression: Efficient storage of large SNMP responses
- Hot/Cold Data Management: Automatic promotion of frequently accessed data
- Cache Warming: Proactive loading of expected data

## Caching Strategies

### Time-Based Caching

Standard TTL-based caching for static or slowly changing data.

### Volatility-Based Caching

Dynamic TTL adjustment based on observed change frequency.

### Dependency-Based Caching

Cache invalidation based on related data changes.

### Predictive Caching

Pre-loading data based on access patterns and time of day.

## Performance Benefits

- 50-80% reduction in redundant SNMP queries
- Improved response times for frequently accessed data
- Reduced network load on monitored devices
- Better scalability for large device inventories

## Usage Patterns

    # Cache SNMP response data
    SnmpKit.SnmpLib.Cache.put("device_123:sysDescr", response_data, ttl: 300_000)

    # Retrieve cached data
    case SnmpKit.SnmpLib.Cache.get("device_123:sysDescr") do
      {:ok, data} -> data
      :miss -> perform_snmp_query()
    end

    # Cache with adaptive TTL
    SnmpKit.SnmpLib.Cache.put_adaptive("device_123:ifTable", interface_data,
      base_ttl: 60_000,
      volatility: :medium
    )

    # Warm cache for predictable access
    SnmpKit.SnmpLib.Cache.warm_cache("device_123", [:sysDescr, :sysUpTime, :ifTable])

    # Invalidate related caches
    SnmpKit.SnmpLib.Cache.invalidate_pattern("device_123:*")

## Cache Key Patterns

- `device_id:oid` - Single OID values
- `device_id:table:index` - Table row data
- `device_id:walk:base_oid` - Walk results
- `device_id:bulk:oids` - Bulk query results
- `global:topology` - Cross-device topology data

# Summary

## Types

cache_key()
cache_opts()
cache_stats()
cache_strategy()
cache_ttl()
cache_value()
volatility()

## Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

clear()

Clears all cached data.

delete(key)

Removes a specific key from the cache.

get(key)

Retrieves a value from the cache.

get_stats()

Gets comprehensive cache performance statistics.

invalidate_by_tag(tag)

Invalidates cache entries by tag.

invalidate_pattern(pattern)

Invalidates multiple cache entries matching a pattern.

put(key, value, opts \\ [])

Stores a value in the cache with specified options.

put_adaptive(key, value, base_ttl, volatility)

Stores a value with adaptive TTL based on observed volatility.

start_link(opts \\ [])

Starts the cache manager with specified configuration.

warm_cache(device_id, oids, opts \\ [])

Pre-loads cache with expected data to improve response times.

# Types

# cache_key()

    @type cache_key() :: binary()

# cache_opts()

    @type cache_opts() :: [
      ttl: cache_ttl(),
      strategy: cache_strategy(),
      volatility: volatility(),
      compress: boolean(),
      dependencies: [cache_key()],
      tags: [atom()]
    ]

# cache_stats()

    @type cache_stats() :: %{
      total_entries: non_neg_integer(),
      hit_rate: float(),
      miss_rate: float(),
      eviction_count: non_neg_integer(),
      memory_usage_mb: float(),
      compression_ratio: float()
    }

# cache_strategy()

    @type cache_strategy() ::
      :time_based | :volatility_based | :dependency_based | :predictive

# cache_ttl()

    @type cache_ttl() :: pos_integer()

# cache_value()

    @type cache_value() :: any()

# volatility()

    @type volatility() :: :low | :medium | :high | :extreme

# Functions

# child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See `Supervisor`.

# clear()

    @spec clear() :: :ok

Clears all cached data.

## Examples

    :ok = SnmpKit.SnmpLib.Cache.clear()

# delete(key)

    @spec delete(cache_key()) :: :ok

Removes a specific key from the cache.

## Examples

    :ok = SnmpKit.SnmpLib.Cache.delete("device_1:sysDescr")

# get(key)

    @spec get(cache_key()) :: {:ok, cache_value()} | :miss

Retrieves a value from the cache.

## Returns

- `{:ok, value}`: Cache hit with the stored value
- `:miss`: Cache miss, value not found or expired

## Examples

    case SnmpKit.SnmpLib.Cache.get("device_1:sysDescr") do
      {:ok, description} ->
        Logger.debug("Cache hit for system description")
        description
      :miss ->
        Logger.debug("Cache miss, performing SNMP query")
        perform_snmp_get(device, [1,3,6,1,2,1,1,1,0])
    end

# get_stats()

    @spec get_stats() :: cache_stats()

Gets comprehensive cache performance statistics.

## Returns

Statistics including hit rates, memory usage, and performance metrics.

## Examples

    cache_stats = SnmpKit.SnmpLib.Cache.get_stats()
    IO.puts "Cache hit rate: " <> Float.to_string(Float.round(cache_stats.hit_rate * 100, 2)) <> "%"
    IO.puts "Memory usage: " <> Float.to_string(Float.round(cache_stats.memory_usage_mb, 2)) <> " MB"

# invalidate_by_tag(tag)

    @spec invalidate_by_tag(atom()) :: :ok

Invalidates cache entries by tag.

## Examples

    SnmpKit.SnmpLib.Cache.invalidate_by_tag(:routing_data)

# invalidate_pattern(pattern)

    @spec invalidate_pattern(binary()) :: :ok

Invalidates multiple cache entries matching a pattern.

Supports wildcards (*) for pattern matching.

## Examples

    # Invalidate all data for a device
    SnmpKit.SnmpLib.Cache.invalidate_pattern("device_1:*")

    # Invalidate all interface data
    SnmpKit.SnmpLib.Cache.invalidate_pattern("*:ifTable")

    # Invalidate by tag
    SnmpKit.SnmpLib.Cache.invalidate_by_tag(:interface_data)

# put(key, value, opts \\ [])

    @spec put(cache_key(), cache_value(), cache_opts()) :: :ok

Stores a value in the cache with specified options.

## Parameters

- `key`: Unique cache key
- `value`: Data to cache
- `opts`: Caching options

## Options

- `ttl`: Time-to-live in milliseconds (default: 300,000)
- `strategy`: Caching strategy (default: :time_based)
- `volatility`: Data change frequency (default: :medium)
- `compress`: Force compression for this entry (default: auto)
- `dependencies`: Keys that invalidate this entry when changed
- `tags`: Metadata tags for grouping and invalidation

## Examples

    # Simple time-based caching
    :ok = SnmpKit.SnmpLib.Cache.put("device_1:sysDescr", "Cisco Router", ttl: 600_000)

    # Adaptive caching based on volatility
    :ok = SnmpKit.SnmpLib.Cache.put("device_1:ifTable", interface_data,
      strategy: :volatility_based,
      volatility: :high,
      tags: [:interface_data]
    )

    # Dependency-based caching
    :ok = SnmpKit.SnmpLib.Cache.put("device_1:route_summary", summary_data,
      dependencies: ["device_1:routeTable", "device_1:arpTable"]
    )

# put_adaptive(key, value, base_ttl, volatility)

    @spec put_adaptive(cache_key(), cache_value(), cache_ttl(), volatility()) :: :ok

Stores a value with adaptive TTL based on observed volatility.

The cache automatically adjusts TTL based on how frequently the data changes.

## Parameters

- `key`: Cache key
- `value`: Data to cache
- `base_ttl`: Starting TTL value
- `volatility`: Expected change frequency

## Examples

    # Interface counters change frequently
    SnmpKit.SnmpLib.Cache.put_adaptive("device_1:ifInOctets", counter_data,
      base_ttl: 30_000,
      volatility: :high
    )

    # System description rarely changes
    SnmpKit.SnmpLib.Cache.put_adaptive("device_1:sysDescr", description,
      base_ttl: 3_600_000,
      volatility: :low
    )

# start_link(opts \\ [])

    @spec start_link(keyword()) :: {:ok, pid()} | {:error, any()}

Starts the cache manager with specified configuration.

## Options

- `max_size`: Maximum number of cache entries (default: 100,000)
- `cleanup_interval`: Cleanup frequency in milliseconds (default: 60,000)
- `compression_enabled`: Enable compression for large values (default: true)
- `adaptive_ttl_enabled`: Enable adaptive TTL based on volatility (default: true)
- `predictive_enabled`: Enable predictive caching (default: false)

## Examples

    # Start with defaults
    {:ok, _pid} = SnmpKit.SnmpLib.Cache.start_link()

    # Start with custom configuration
    {:ok, _pid} = SnmpKit.SnmpLib.Cache.start_link(
      max_size: 50_000,
      compression_enabled: true,
      predictive_enabled: true
    )

# warm_cache(device_id, oids, opts \\ [])

    @spec warm_cache(binary(), [binary()] | :auto, keyword()) :: :ok

Pre-loads cache with expected data to improve response times.

## Parameters

- `device_id`: Target device identifier
- `oids`: List of OIDs to pre-load
- `strategy`: Warming strategy (:immediate, :scheduled, :predictive)

## Examples

    # Immediate cache warming
    SnmpKit.SnmpLib.Cache.warm_cache("device_1",
      ["1.3.6.1.2.1.1.1.0", "1.3.6.1.2.1.1.3.0"],
      strategy: :immediate
    )

    # Predictive warming based on historical access
    SnmpKit.SnmpLib.Cache.warm_cache("device_1", :auto,
      strategy: :predictive
    )