Pixir.Compaction (pixir v0.1.14)

Copy Markdown View Source

Durable History compaction.

Pixir keeps the full Log as source of truth, but the Provider does not need every old Event on every Turn. Compaction records a canonical history_compaction Event that summarizes an older prefix and leaves a recent tail uncompressed. Provider replay then sends the latest checkpoint plus the tail, so context stays bounded while resume/fork/debug still read the original Log.

Summary

Functions

Capturing identity used to persist and fold a native window.

Append a durable history_compaction checkpoint when there is compactable History.

Product compact_threshold sent on overlay-on ordinary Turns.

OpenAI Responses validation minimum for compact_threshold.

True when a prior Turn already recorded a compact_threshold rejection for the current checkpoint range. Prevents retrying the field every Turn.

Run compaction and normalize its terminal outcome for runtime presenters.

Default tail size used for recovery/preflight compactions.

Developer instruction for a future model-assisted compaction pass.

Plan compaction without appending anything to the Log.

Last seq of History actually sent as Provider input after provider_history/1.

Bounded inspect of native_replay: mode, usable, ids, fallback reason.

The to_seq of the latest compaction checkpoint in history, or nil when the Session has never been compacted. Keys context-pressure warning hysteresis (ADR 0020): a new checkpoint re-arms the warning gate.

Build the instruction, schema, and delimited user payload for model-assisted compaction. This does not call the network and does not mutate the Log.

Operator overlay preference for C and D.

True when Provider fold may send native_replay.items as the compacted prefix.

Local history_compaction data for a mid-Turn native_threshold fire.

JSON schema for a model-assisted compaction checkpoint.

Overlay bit after Provider/backend resolve.

Attach a standalone_window from a compact output list.

Persist a live threshold capture, always keeping local text.

Attach a threshold_item native replay window to local checkpoint data.

Build the deterministic compaction plan for a Session.

Replace checkpoint native_replay with the inspect projection.

Sanitize compact/complete JSON so ciphertext never prints.

Return the Provider-visible History: latest compaction checkpoint plus uncompressed events after its range. Sessions without compaction pass through unchanged.

Render compaction data as a Provider input item.

Validate native_replay for threshold_item or standalone_window.

Functions

capturing_current(resolved)

@spec capturing_current(
  map()
  | Pixir.Providers.ResolvedProviderRequest.t()
  | keyword()
) :: map()

Capturing identity used to persist and fold a native window.

compact(session_id, opts \\ [])

@spec compact(
  String.t(),
  keyword()
) :: {:ok, map()} | {:error, map()}

Append a durable history_compaction checkpoint when there is compactable History.

compact_threshold()

@spec compact_threshold() :: pos_integer()

Product compact_threshold sent on overlay-on ordinary Turns.

This is 200_000, not OpenAI's API minimum. Native D should fire around advisory pressure; local 90% critical preflight / overflow / websocket recovery stay the backstop.

compact_threshold_minimum()

@spec compact_threshold_minimum() :: pos_integer()

OpenAI Responses validation minimum for compact_threshold.

Values below 1000 are rejected by the API. The product default is compact_threshold/0; this accessor exists so a future override cannot send below the floor.

compact_threshold_suppressed?(history)

@spec compact_threshold_suppressed?([Pixir.Event.t()]) :: boolean()

True when a prior Turn already recorded a compact_threshold rejection for the current checkpoint range. Prevents retrying the field every Turn.

complete(session_id, opts \\ [])

@spec complete(
  String.t(),
  keyword()
) :: {:ok, map()}

Run compaction and normalize its terminal outcome for runtime presenters.

The result always distinguishes recorded, no_op, and error, and carries the compacted range/checkpoint only when they exist. It deliberately has no token-pressure field: local deterministic compaction does not call a Provider or own a tokenizer, so it cannot honestly calculate the post-compaction used gauge.

default_tail_events()

@spec default_tail_events() :: {:ok, pos_integer()}

Default tail size used for recovery/preflight compactions.

developer_instruction()

@spec developer_instruction() :: String.t()

Developer instruction for a future model-assisted compaction pass.

Keep this short and contract-like. Detailed shape belongs in output_schema/0, and actual Session facts belong in the per-call input payload.

dry_run(session_id, opts \\ [])

@spec dry_run(
  String.t(),
  keyword()
) :: {:ok, map()} | {:error, map()}

Plan compaction without appending anything to the Log.

input_to_seq(history)

@spec input_to_seq([Pixir.Event.t()]) :: non_neg_integer() | nil

Last seq of History actually sent as Provider input after provider_history/1.

This is the frozen input_to_seq for a native_threshold checkpoint.

inspect_native_replay(data)

@spec inspect_native_replay(term()) :: map() | nil

Bounded inspect of native_replay: mode, usable, ids, fallback reason.

Never includes encrypted_content or verbatim items.

latest_checkpoint_to_seq(history)

@spec latest_checkpoint_to_seq([Pixir.Event.t()]) :: non_neg_integer() | nil

The to_seq of the latest compaction checkpoint in history, or nil when the Session has never been compacted. Keys context-pressure warning hysteresis (ADR 0020): a new checkpoint re-arms the warning gate.

model_contract(session_id, events, opts \\ [])

@spec model_contract(String.t(), [Pixir.Event.t()], keyword()) ::
  {:ok, map()} | {:error, map()}

Build the instruction, schema, and delimited user payload for model-assisted compaction. This does not call the network and does not mutate the Log.

native_preference(opts \\ [])

@spec native_preference(keyword()) :: nil | boolean()

Operator overlay preference for C and D.

nil means no preference (default-on after resolve for chatgpt_codex and official api.openai.com). Explicit false always wins. One key — not a second threshold switch. Standalone C still requires a host that serves /responses/compact; on chatgpt_codex C stays local.

native_replay_fold_usable?(event_data, current)

@spec native_replay_fold_usable?(map(), map()) :: boolean()

True when Provider fold may send native_replay.items as the compacted prefix.

Anthropic fold never takes this branch.

native_threshold_event_data(history, input_to_seq, item, opts \\ [])

@spec native_threshold_event_data(
  [Pixir.Event.t()],
  non_neg_integer() | nil,
  term(),
  keyword()
) ::
  {:ok, map()} | {:error, map()}

Local history_compaction data for a mid-Turn native_threshold fire.

range.to_seq is the frozen input_to_seq, not Log tip. Local text fields remain mandatory. item is the latest stream cmp_ (or nil on failed capture).

output_schema()

@spec output_schema() :: map()

JSON schema for a model-assisted compaction checkpoint.

This intentionally lives in code instead of prompt prose so the Provider path can enforce the shape with structured outputs when model-assisted compaction is enabled.

overlay_after_resolve(preference, resolved)

@spec overlay_after_resolve(
  nil | boolean(),
  map() | Pixir.Providers.ResolvedProviderRequest.t()
) ::
  {:on, nil} | {:off, String.t()}

Overlay bit after Provider/backend resolve.

preference is nil (no preference), true (request on), or false.

persist_standalone_window(local_event_data, output, opts \\ [])

@spec persist_standalone_window(map(), term(), keyword()) ::
  {:ok, map()} | {:error, map()}

Attach a standalone_window from a compact output list.

Persists the entire unpruned output. Local text fields remain mandatory.

persist_threshold_capture(local_event_data, item_or_items, opts \\ [])

@spec persist_threshold_capture(map(), term(), keyword()) ::
  {:ok, map()} | {:error, map()}

Persist a live threshold capture, always keeping local text.

A usable singleton cmp_ becomes recorded_usable true. Failed or missing capture still returns local checkpoint data with recorded_usable false and a stable fallback_reason.

persist_threshold_item(local_event_data, item_or_items, opts \\ [])

@spec persist_threshold_item(map(), map() | [map()], keyword()) ::
  {:ok, map()} | {:error, map()}

Attach a threshold_item native replay window to local checkpoint data.

item_or_items must be exactly the latest cmp_ compaction item. A full compact output list is rejected as threshold_item_not_singleton. Live Turn ingest uses native_threshold_event_data/4.

plan(session_id, opts \\ [])

@spec plan(
  String.t(),
  keyword()
) :: {:ok, map()} | {:error, map()}

Build the deterministic compaction plan for a Session.

project_checkpoint_for_inspect(data)

@spec project_checkpoint_for_inspect(map()) :: map()

Replace checkpoint native_replay with the inspect projection.

project_compact_result_for_inspect(result)

@spec project_compact_result_for_inspect(map()) :: map()

Sanitize compact/complete JSON so ciphertext never prints.

provider_history(history)

@spec provider_history([Pixir.Event.t()]) :: [Pixir.Event.t()]

Return the Provider-visible History: latest compaction checkpoint plus uncompressed events after its range. Sessions without compaction pass through unchanged.

render_for_provider(data)

@spec render_for_provider(map()) :: String.t()

Render compaction data as a Provider input item.

validate_native_replay(native_replay, opts \\ [])

@spec validate_native_replay(
  term(),
  keyword()
) :: {:ok, map()} | {:error, map()}

Validate native_replay for threshold_item or standalone_window.

Pass compact_output: so reducing that output to cmp_ alone is standalone_window_pruned.