Scoped long-term memory

View Source

Erlang ADK 0.5 uses version 2 of adk_memory_service. Long-term memory is always authorized by an explicit application/user scope:

{user, AppName, UserId}

The same user identifier in two applications is two independent principals. Metadata cannot widen this scope. Entries contain bounded UTF-8 content, JSON-safe metadata, optional session/event provenance, a stable identifier, SHA-256 digest, and timestamp. Memory is reference data rather than executable instructions; Runner frames retrieved text as untrusted input before it reaches a model.

Long-term memory and its Runner retrieval/ingestion path were already released in 0.9. The 0.10 branch adds an opt-in supervised configuration layer: adk_runtime_service_bundle:start_link(ephemeral_local, #{}) selects one shared ETS memory adapter, while durable_local selects exact-scope local Mnesia workers. The shared adapter enforces one global quota across users; durable limits remain per worker/shard. The bundle returns the same native memory service reference. durable_local also atomically owns a private Mnesia ingestion outbox, validates its adapter registration and health, and injects durable ingestion into standard Runner options. Pending jobs survive a bundle process restart, while stale or unhealthy references fail closed. Disabled and ephemeral_local profiles do not create this private outbox; the standalone opt-in described below remains compatible. Separately, 0.10 adds bounded embedding and vector/hybrid contracts, an opt-in governance hook, durable erasure epochs, and terminal outbox retention. The included vector adapter is local and volatile, not a managed or distributed vector database. See VERSION_0_10_0.md; 0.10 is still IN DEVELOPMENT.

ETS and Mnesia adapters

adk_memory_ets is volatile and intended for tests or local development. adk_memory_mnesia uses disc_copies and preserves entries across a normal VM restart when the node name and Mnesia directory are stable. Both implement the same deterministic lexical-overlap search contract:

{ok, MemoryPid} = adk_memory_ets:start_link(#{}),
Scope = {user, <<"my_app">>, <<"user-42">>},

{ok, Entry} = adk_memory_ets:add_entry(
    MemoryPid, Scope,
    #{content => <<"OTP supervisors restart failed child processes">>,
      metadata => #{<<"topic">> => <<"otp">>},
      provenance => #{session_id => <<"session-7">>,
                      author => <<"user">>,
                      timestamp => erlang:system_time(millisecond)}},
    #{idempotency_key => <<"session-7:fact-1">>}),

{ok, [Hit]} = adk_memory_ets:search(
    MemoryPid, Scope, <<"supervisors restart">>,
    #{filter => #{<<"topic">> => <<"otp">>}, limit => 5}),
EntryId = maps:get(id, Entry),
EntryId = maps:get(id, Hit),

ok = adk_memory_ets:delete_entry(
    MemoryPid, Scope, EntryId),
ok = adk_memory_ets:stop(MemoryPid).

Use the same calls with adk_memory_mnesia. The adapter starts Mnesia and creates its two local tables if needed:

{ok, DurableMemoryPid} = adk_memory_mnesia:start_link(#{}),
#{contract_version := 2, durable := true} =
    adk_memory_mnesia:capabilities(DurableMemoryPid).

In a release, configure Mnesia's directory and distributed table-copy policy before starting this adapter. The bundled implementation is a durable local lexical reference adapter, not a distributed vector database. Applications that need a managed index, multi-region replication, or provider-specific retention should implement the corresponding service/vector/policy contracts.

Both adapters reject unknown options and expose configured limits through capabilities/1. Deadline-aware mutations accept #{timeout_ms => PositiveMilliseconds} as their final argument. The deadline travels with queued work, so an expired request cannot later become an invisible write:

adk_memory_ets:add_entry(
    MemoryPid, Scope, #{content => <<"bounded fact">>}, #{},
    #{timeout_ms => 2000}).

Embeddings and vector/hybrid retrieval (0.10)

adk_memory_embedding_provider:embed/5 invokes an application-selected embedding provider in a monitored, killable worker. The request and reply are validated before crossing the boundary. Defaults cap a call at 5 seconds, 128 inputs, 64 KiB per input, 1 MiB total input, 8,192 dimensions, and 16 MiB of vector result storage. A provider must return the requested model and exactly one finite, fixed-width vector per input; credentials and endpoints remain in its trusted handle.

adk_memory_vector_ets is the bounded local reference implementation of adk_memory_vector_adapter and adk_memory_hybrid_adapter. It supports app/user-scoped batch upsert, cosine-similarity search, weighted lexical/vector hybrid search, exact-scope deletion, and content-free status. It enforces entry, byte, dimension, batch, result-count, and result-byte ceilings. All data is volatile and one instance fixes one vector dimension after its first upsert.

These contracts are not silently substituted for adk_memory_ets or adk_memory_mnesia, and merely configuring memory_svc does not call an embedding provider. Applications own embedding refresh, index synchronization, managed-vector adapters, and any bridge from vector hits into Runner retrieval.

Exact-user sharded concurrency

adk_memory_sharded implements the same version-2 behavior. Its default exact_scope strategy uses one active supervised adapter worker per exact {user, App, User} scope. Same-user calls remain ordered by that worker, while unrelated users execute concurrently after a protected ETS routing fast path:

{ok, ShardedMemory} = adk_memory_sharded:start_link(
    #{adapter => adk_memory_mnesia,
      adapter_config => #{},
      max_active_scopes => 1024,
      max_router_queue => 256}),
#{contract_version := 2,
  durable := true,
  quota_scope := exact_scope_shard,
  global_quota := false} =
    adk_memory_sharded:capabilities(ShardedMemory),
{ok, _} = adk_memory_sharded:add_entry(
    ShardedMemory, {user, <<"my_app">>, <<"user-42">>},
    #{content => <<"OTP isolates failures">>},
    #{idempotency_key => <<"user-42:otp-fact">>}),
ok = adk_memory_sharded:stop(ShardedMemory).

The default adapter is adk_memory_ets; select adk_memory_mnesia when local restart durability is required. The wrapper validates the complete v2 callback set before startup and can be passed to Runner as {adk_memory_sharded, ShardedMemory} or registered as a durable-outbox adapter. Owner death and explicit stop/1 clean up the per-instance dynamic supervisor and its children. A failed worker is removed and recreated on the next call; volatile ETS data is lost with that worker, while Mnesia data remains in the scoped tables.

max_active_scopes bounds worker cardinality, while max_router_queue strictly caps simultaneous cold-scope resolutions before they enter the router. A guard independently monitors each unresolved caller, owns and releases its permit on death or timeout, and prevents a stale queued route from creating an abandoned worker. Resolved scopes use the protected ETS fast path.

For the default volatile ETS adapter, idle reclamation is disabled because evicting a worker would discard its entries. Its active-scope ceiling is thus a lifetime/cardinality bound until a worker exits or the router restarts. With the durable Mnesia adapter, the router may reclaim the least-recently-used worker at capacity after all of its operation leases are idle for idle_scope_timeout_ms (default 60000; range 1 through 86400000). The next worker for that exact user reopens the same Mnesia data. If no worker is eligible, a new scope returns {error, max_active_scopes_reached}.

Exact-scope limits and quotas belong to each shard, not one aggregate service budget, and capabilities report quota_scope => exact_scope_shard and global_quota => false. A direct wrapper can instead set scope_strategy => shared, creating one adapter instance for every authorized scope, reporting active_scopes => 1, and enforcing its limits globally as shared_adapter. The 0.10 ephemeral_local profile uses this shared shape. Status exposes routing, idle timeout, whether reclamation is active, and the eviction count; sharding capabilities expose the strategy, timeout, and reclamation mode. Use an outer admission controller or custom backend for a deployment-wide quota over durable exact-scope shards.

Incremental ingestion and erasure

add_events/5 sanitizes canonical session events, creates deterministic idempotency keys, and reports additions, duplicates, and skipped events:

{ok, Session} = erlang_adk_session:get_session(
    <<"my_app">>, <<"user-42">>, <<"session-7">>),
{ok, #{added := Added, duplicates := Duplicates, skipped := Skipped}} =
    adk_memory_ets:add_events(
      MemoryPid, Scope, <<"session-7">>, maps:get(events, Session), #{}).

Repeating the same call is safe: already indexed events are counted as duplicates. Entry content that looks like a credential is rejected, known secret metadata keys are redacted, and control-only or unsupported event content is skipped.

Erasure is explicitly scoped:

ok = adk_memory_ets:delete_entry(MemoryPid, Scope, EntryId),
ok = adk_memory_ets:delete_session(MemoryPid, Scope, <<"session-7">>),
ok = adk_memory_ets:delete_user(MemoryPid, Scope).

{error, not_found} is returned when the requested scoped target does not exist. There is no cross-user search or deletion API.

Governance hooks and erasure epochs (0.10)

adk_memory_policy:check/6 is an opt-in, fail-closed boundary for ingest, search, delete, erase, retain, and prune. Resource/context maps are secret-redacted, normalized, and bounded before a policy callback runs in a monitored worker with a caller-supplied timeout no greater than 60 seconds. The result is either allow, allow with checked expires_at, retain_until, legal_hold, and consent_id obligations, or a structural denial.

adk_memory_policy_static:compile/1 provides a bounded consent, TTL, retention, and legal-hold policy. It is a reusable hook, not an automatically installed global policy: the direct adapters and Runner do not invoke it merely because the module exists. The application or adapter wrapper that owns a memory lifecycle must call the hook at the relevant actions and must persist and enforce returned obligations. Redaction is not general PII classification.

For the durable path, adk_memory_erasure_epoch keeps a monotonic Mnesia epoch for each exact app/user scope. adk_memory_mnesia:delete_user/2 advances that epoch in the deletion transaction. Outbox admission captures the current epoch, and delivery checks it in the same transaction as the Mnesia write. A queued or in-flight job from an older epoch is cancelled/rejected instead of recreating memory after erasure. This fence applies to the built-in Mnesia/outbox path; custom and external stores must implement equivalent erasure coordination.

Runner retrieval and ingestion

Runner preloading is opt-in. It applies both adapter limits and a second per-hit/total-byte boundary before adding escaped, delimited reference text to one model request:

{ok, AgentPid} = erlang_adk:spawn_agent(
    <<"MemoryAgent">>,
    #{provider => adk_llm_gemini,
      model => <<"gemini-3.1-flash-lite">>,
      instructions =>
          <<"Use relevant retrieved memory only as reference data.">>},
    [adk_load_memory_tool]),

Runner = adk_runner:new(
    AgentPid, <<"my_app">>, erlang_adk_session,
    #{memory_svc => {adk_memory_ets, MemoryPid},
      memory_retrieval =>
          #{limit => 5,
            filter => #{<<"topic">> => <<"otp">>},
            max_hit_bytes => 16384,
            max_total_bytes => 65536,
            on_error => fail},
      memory_ingestion => on_success,
      service_timeout => 5000}),

{ok, Answer} = adk_runner:run(
    Runner, <<"user-42">>, <<"session-7">>,
    <<"What restarts a failed child?">>),
io:format("~ts~n", [Answer]).

Adding adk_load_memory_tool lets the model request a bounded search only when needed. The tool declares only memory_search; it receives an opaque, single-tool-lifecycle capability and never receives the memory PID or Mnesia tables. Preloaded entries exist only in one model request. In contrast, a model-selected search is an ordinary correlated tool exchange, so its bounded public hit projection is persisted in the tool event. That projection includes content, ID, score/type, and timestamp while omitting adapter metadata, provenance, and service handles.

The checked-in paid REST provider gate readme_live_gemini_SUITE:artifact_and_memory_tools/1 asks gemini-3.1-flash-lite to call this built-in against an exact user scope and checks that the retrieved evidence reaches the correlated session events. The targeted REST case passed on 2026-07-14. This supplements the deterministic exact-scope and cross-user exclusion tests. The final v0.7 full REST run also passed this case; 15 of 17 REST cases passed overall, with the two failures caused by HTTP 429 in Search grounding and cached-content creation.

The built-in declaration contains strict JSON Schema constraints, including additionalProperties. At the Gemini boundary it is therefore emitted as parametersJsonSchema, while Erlang ADK retains the same compiled schema for local argument validation. The targeted REST pass exercises that projection.

memory_ingestion => on_success returns the final answer without waiting for indexing. The application supervisor runs ingestion in bounded workers, splits large event lists into idempotent batches, and performs bounded retries. The default Runner worker queue is process-local: successful writes are durable when the adapter is durable, but work not yet admitted to the adapter is not restart-proof.

Durable ingestion outbox

Applications that require restart-safe admission can use the Mnesia-backed adk_memory_outbox and a bounded processor. The durable job stores sanitized canonical events, an exact scope/session, batch checkpoints, and a stable {AdapterModule, AdapterId}. It never persists a PID, service handle, credential, or resolver state:

{ok, Outbox} = adk_memory_outbox:init(#{}),
{ok, Registry} = adk_memory_outbox_registry:start_link(),
MemoryService = {adk_memory_mnesia, DurableMemoryPid},
AdapterIdentity = {adk_memory_mnesia, <<"primary-memory">>},
ok = adk_memory_outbox_registry:register(
    Registry, AdapterIdentity, MemoryService),
{ok, Processor} = adk_memory_outbox_processor:start_link(
    #{outbox => Outbox,
      resolver => {adk_memory_outbox_registry, Registry},
      max_concurrency => 4,
      call_timeout_ms => 5000,
      lease_ms => 15000}),

{ok, Job} = adk_memory_outbox_processor:submit(
    Processor,
    #{scope => Scope,
      session_id => <<"session-7">>,
      adapter => AdapterIdentity,
      events => maps:get(events, Session),
      max_attempts => 5}),
JobId = maps:get(job_id, Job),
{ok, JobStatus} = adk_memory_outbox_processor:status(Processor, JobId).

Multiple processors may share the tables. Claims use an unguessable ownership token and a bounded lease time; after a worker or node failure another processor can retry the batch. Stable adapter resolution runs in a monitored, timeout-bounded worker that is killed and drained on timeout. Capability discovery is also bounded. After both finish and immediately before add_events, the processor renews the lease and revalidates that the original token still owns the job. A lost or expired owner does not begin the adapter mutation. Delivery remains at-least-once, so the processor requires an adapter advertising contract_version >= 2, idempotent_ingestion, and incremental_events; add_events/5 is required and deadline-aware add_events/6 is preferred. Stable event IDs make a repeated batch a duplicate rather than a second memory entry.

The volatile registry is also an admission barrier. On initial startup and after a registry child restart, processors claim no durable job until the application has deterministically re-registered at least one adapter. The registry then returns only the bounded set of hydrated stable identities, so registering adapter A cannot release queued work for unavailable adapter B. Claim discovery scans at most max_claim_scan entries in the ordered schedule index and persists a rotating cursor; an unavailable identity at the front cannot permanently starve later hydrated identities. Resolver readiness and the identity snapshot run in bounded monitored workers and malformed, oversized, timed-out, or failed replies close the barrier.

Jobs and total bytes have global and per-scope admission limits. Events are size/count bounded and sanitized before the enqueue transaction. Retry uses bounded exponential backoff and becomes terminal after max_attempts. adk_memory_outbox_processor:status/2 and the lower-level adk_memory_outbox:stats/1, cancel/3, and terminal-job delete/2 expose explicit lifecycle operations without returning content or runtime handles.

The ordered schedule table indexes pending/retry due times, running-lease expiry, and terminal completion time. Claims, expired-lease recovery, erasure- epoch fencing, and terminal pruning therefore scan explicit bounded index windows instead of folding the jobs table. Epoch fencing is rechecked inside claim and immediately before mutation renewal.

Terminal capacity is a hard reservation across active plus terminal jobs: every admitted active job reserves one eventual terminal slot. Defaults retain terminal history for seven days, cap the combined reservation at 100,000 records, and cap one prune pass at 1,000 records. A pre-cap database that is already over the configured ceiling remains startable for migration and bounded pruning, but new admissions fail closed until delete/prune creates headroom. Use adk_memory_outbox:prune_terminal/2,3, adk_memory_outbox_processor:prune_terminal/2, or the supervised adk_memory_outbox_sup:prune_terminal/1 API to remove only terminal rows older than the configured retention cutoff. Terminal counts and the ordered index are rebuilt during migration from older jobs-only data. Pruning is explicit and bounded; there is no timer that silently deletes history.

The batch ceiling is 500 events and the default job ceiling is 5,000 events. The processor validates lease_ms >= 2 * call_timeout_ms + 250 to leave a bounded resolution/capability phase and a fresh adapter-call lease. The token fences outbox state transitions, but it is not a true generation fence inside an arbitrary adapter: a lease can expire after the final pre-call check while adapter code is running. Adapters must honor the supplied deadline where available, and correctness across a retry relies on stable event-ID idempotency. Job-level deduplication requires an exact adapter, scope, session, and ordered event-ID sequence in one erasure epoch; the epoch is part of the durable job identifier. Repeating the same request in the same epoch is idempotent. After successful erasure advances the epoch, the same logical request receives a new job ID and may be admitted again without reviving pre- erasure work. Partially overlapping jobs rely on the v2 adapter's event-ID idempotency. This is deliberately not an exactly-once outbox.

The outbox registry must re-register the stable adapter identity after an adapter restart. Pending jobs remain durable while resolution is unavailable and retry according to policy.

With the durable_local runtime profile, adk_runtime_service_bundle owns a private outbox supervisor in the same atomic generation as the session, artifact, and memory services. Startup validates the selected memory adapter's durable-ingestion capabilities—including numeric contract_version >= 2, idempotent/incremental ingestion, and erasure-epoch fencing—before exposing the bundle. Unknown or invalid nested outbox, registry, and processor options fail profile compilation; names are forced private and Runner attempts remain within its hard ceiling. services/1, runner_spec/1, status/1, and health/1 expose validated, redacted outbox service state; runner_spec/1 injects the required memory_ingestion map into standard Runner creation. Outbox Mnesia jobs survive a bundle/outbox process restart and resume after the stable adapter is re-registered. A stale bundle generation, dead private outbox, unhealthy registry/processor, or mismatched adapter identity is an error rather than a fallback to process-local writes.

ephemeral_local and the disabled runtime-service profile intentionally own no private outbox. Existing applications may continue to use the standalone memory_outbox_enabled application setting and explicit Runner options below; when durable_local is enabled, legacy module-named adk_memory_outbox_sup APIs resolve the one bundle-owned supervisor rather than starting or addressing a duplicate processor.

Outbox health is independent of retained-job cardinality. It validates the jobs, usage, ordered schedule, and erasure-epoch table schemas/topology, then performs fixed point reads and one sentinel write in a single transaction. The sentinel leaves all four table row counts unchanged. mnesia_majority mode fails admission, claims, renewal, and health closed unless the four tables share at least two configured nodes; single-node mode is explicit and does not claim multi-node readiness. Public status and crash formatting omit service handles, event content, resolver state, owner tokens, and private inputs.

Runner can use the standalone compatibility outbox when it is enabled before application startup:

ok = application:set_env(erlang_adk, memory_outbox_enabled, true),
ok = application:set_env(
    erlang_adk, memory_outbox_options,
    #{outbox => #{max_active_per_scope => 1000,
                  max_active_bytes_per_scope => 67108864},
      registry => #{max_entries => 128},
      processor => #{max_concurrency => 4,
                     call_timeout_ms => 5000,
                     lease_ms => 15000}}),
{ok, _} = application:ensure_all_started(erlang_adk).

Select durable admission on the Runner with a stable adapter identifier. The adapter module comes from memory_svc; the PID or handle is registered only in the runtime registry:

DurableRunner = adk_runner:new(
    AgentPid, <<"my_app">>, erlang_adk_session_mnesia,
    #{memory_svc => {adk_memory_mnesia, DurableMemoryPid},
      memory_ingestion =>
          #{mode => durable,
            adapter_id => <<"primary-memory">>,
            max_attempts => 5}}).

on_success remains the lower-latency process-local shorthand. Durable mode performs a bounded local Mnesia admission transaction after the final session events exist, then leaves adapter delivery to the processors. Runner creation fails with memory_outbox_runtime_required if the configured supervisor, registry, or processor is unavailable. If the bounded admission transaction later fails, the run caller receives {error, {durable_memory_ingestion_not_admitted, Reason}} instead of a successful final answer. The already persisted final session event is not rolled back; retry the same logical invocation only under the application's normal invocation-id/idempotency policy.

Successful admission emits [erlang_adk, memory, outbox, admitted]; use adk_memory_outbox_sup:status/1 with the telemetry-correlated job ID for operational status. Admission is the durable boundary, not delivery: a later terminal adapter failure is visible through job status and does not change the already completed model run. Submission waits only for sanitization and the bounded local Mnesia transaction; it never waits for adapter resolution or delivery.

Custom least-authority tools

A local tool opts into only the context operations it needs:

-module(memory_lookup_tool).
-behaviour(adk_tool).

-export([schema/0, context_capabilities/0, execute/2]).

schema() ->
    #{<<"name">> => <<"memory_lookup">>,
      <<"description">> => <<"Find relevant saved preferences">>,
      <<"parameters">> =>
          #{<<"type">> => <<"object">>,
            <<"properties">> =>
                #{<<"query">> => #{<<"type">> => <<"string">>}},
            <<"required">> => [<<"query">>],
            <<"additionalProperties">> => false}}.

context_capabilities() -> [memory_search].

execute(#{<<"query">> := Query}, Context) ->
    adk_context:search_memory(
      Context, Query, #{filter => #{}, limit => 5}).

Available memory declarations are memory_search, memory_add, and memory_delete. A declared-capability tool is projected to public invocation identity plus its opaque token. Operations outside the declaration return {error, {context_capability_denied, Operation}}. Modules without context_capabilities/0 remain on the explicit compatibility path for 0.5; new tools should always declare their authority.

The opaque capability verifies the embedded scope on every successful add and every search hit. One foreign-scope or malformed record rejects the whole adapter response as {error, invalid_memory_service_reply}; it is never partially filtered or relabelled, and a rejected add records no effect.

Developer inspection

The authenticated developer API/CLI exposes bounded search projections for one exact {user, App, User} scope and exact confirmed erasure. It never returns adapter metadata, provenance, handles, or tables. Every returned hit must itself embed the requested scope; a mismatched adapter record is rejected as unavailable instead of being relabelled under the HTTP path scope.

Current limits

  • The primary ETS/Mnesia adk_memory_service adapters search by lexical overlap. adk_memory_vector_ets supplies a separate local vector/hybrid reference contract; managed indexes and automatic synchronization remain adapter/application responsibilities.
  • Direct ETS and Mnesia reference adapters perform storage and lexical ranking in one GenServer per service. The optional router wrapper overlaps unrelated exact user scopes while preserving same-scope ordering, or routes through one shared adapter. Durable exact-scope workers support idle LRU reclamation only at capacity; volatile exact-scope workers are not reclaimed. Exact-scope quotas are not aggregated.
  • The core rejects known secret patterns and keys, but this is not general PII detection. The static governance policy exists but is opt-in; applications must invoke it and enforce its obligations on every relevant path.
  • Explicit entry/session/user erasure is implemented. Built-in Mnesia memory and the durable outbox coordinate user erasure with transactional epochs; ETS, custom, and external backends need their own equivalent fencing.
  • The built-in outbox defaults to local disc_copies. Explicit mnesia_majority mode requires at least two nodes shared by all four tables and otherwise fails closed. Cross-node recovery still needs the deployment's replication, backup, restore, and partition policy; no multi-node node-loss Common Test is claimed. Completed, failed, or cancelled history uses the hard active-plus-terminal reservation and bounded indexed explicit pruning, not an automatic timer.
  • The outbox has lease-owned, idempotent at-least-once delivery, not an adapter-generation fence or exactly-once external side effects.
  • Runner preloading and model-selected loading are independent opt-ins. Merely configuring memory_svc does not change the prompt.