Audience: people building against PtcRunner run evidence — reading it through
analysis/*capabilities, the CLI, or the Viewer, or changing how it is produced.
Status: implemented retained product contract, including the local private inspection artifact.
This page defines what a canonical trace and a private inspection artifact
contain, and how each may be queried. It is the field guide to the record
vocabulary, not a walkthrough: for the evidence graph and the debugging path
through it, read the
debug-navigation reference first.
Owner-process lifecycle, immutable capture, publication mechanics, and the
contract-test inventory are implementation concerns and live in the Kernel
maintainer guide (docs/maintainers/kernel.md) and beside the code in the
PtcRunner.Kernel.TraceLog, PtcRunner.Kernel.EventSink,
PtcRunner.Kernel.InspectionSink, and PtcRunner.Kernel.InspectionArtifact
module documentation.
Producing and opening a capture
Nothing here is queryable until a run writes a trace. A canonical trace alone
answers the public activity collection; every other collection additionally
needs the private inspection artifact, which a run only writes when asked.
Both destinations must already exist: --trace-dir names an existing
directory, and --inspect names a file inside one.
mkdir -p traces traces-private
# Canonical trace only — supports analysis/runs, analysis/open, analysis/counters, and activity.
ptc run PROJECT.json --trace-dir traces/
# Trace plus private inspection — additionally supports every private
# collection: turns, model_exchanges, generated_sources, and the rest.
ptc run PROJECT.json --trace-dir traces/ \
--inspect traces-private/run.ptcins
A project document can capture both instead of the two switches; see project configuration.
Three code-owned profiles read those artifacts. run-analysis-v1 takes the
canonical trace and grants the four analysis-* capabilities (analysis-runs,
analysis-open, analysis-read, analysis-counters).
private-run-analysis-v2 additionally takes the inspection artifact, and
requires an authorized private sink because the records it returns carry exact
prompts, generated source, and capability payloads. private-run-catalog-v1
uses bounded metadata-only probes of both directories and grants only the
profile-local analysis/catalog; it does not admit a run or open an inspection
payload.
# Public: canonical evidence only.
ptc repl --profile run-analysis-v1 --resource traces=traces/
# Private: adds every private collection. --resource takes the DIRECTORY
# holding the artifact, not the .ptcins path --inspect was given.
ptc repl --profile private-run-analysis-v2 --private-terminal \
--resource traces=traces/ --resource inspection=traces-private/
# Private catalog: safe cohort metadata without payload admission.
ptc repl --profile private-run-catalog-v1 --private-terminal \
--resource traces=traces/ --resource inspection=traces-private/
Use --private-unattended in place of --private-terminal when no terminal is
attached, and --load QUERY.clj to run forms non-interactively. ptc repl --describe-profile NAME prints a profile's static contract.
Purpose and boundary
TraceLog preserves bounded, queryable facts about Kernel runs for humans, automated diagnostics, and explicitly authorized PTC-Lisp workflows.
Kernel and capability providers
emit canonical bounded events
TraceLog
stores, sanitizes, loads, indexes, filters, and bounds event queries
Consumers
ptc_viewer, CLI/debug tools, and the shared analysis/* capability surfaceKernel does not know how traces are stored or queried. TraceLog does not control workflow execution. Consumers do not invent competing event representations.
Canonical events are the only plane that reconstructs a run. The host logger
and :telemetry carry sparse operational diagnostics and low-cardinality
measurements; neither is an event store or an authorization boundary, and
neither ever carries prompts, generated source, capability arguments or
results, credentials, transport headers, session IDs, or endpoints. Exact
application payloads appear only in an explicitly enabled private inspection
artifact. The Kernel maintainer guide owns the full plane taxonomy.
The Viewer has one canonical event model. The inspection loader is not a replacement trace schema; it joins private records to canonical IDs.
Canonical authority
The append-only canonical event stream is authoritative. Derived run metadata, counters, indexes, and viewer projections must be rebuildable from it.
An index may accelerate discovery and pagination, but it:
- is never the sole copy of run facts;
- records the source identity/version it indexes;
- is invalidated or rebuilt when its source changes;
- does not silently merge unrelated trace sources;
- cannot grant access beyond the underlying source grant.
Do not maintain a second authoritative session/run database.
Storage sources
The current trace schema supports:
- append-only JSONL files;
- explicitly granted directories containing JSONL files;
- a bounded in-memory sink for REPL, tests, and short-lived runs.
JSONL files are written in canonical sequence order. A generated trace
directory is a set of filename-bound canonical files:
<run-id>.jsonl for sanitized evidence and
<run-id>.private.jsonl for private evidence. Each selected directory member
contains exactly one run ID and trace ID, and its embedded run ID equals the
filename stem. The stem is 1–256 ASCII bytes matching
[A-Za-z0-9][A-Za-z0-9._-]*. Directory admission bounds and sorts the raw
selected basenames, inventories every selected member before reading any,
proves the root/name/file/content selection stable, and classifies a complete
run/trace claim graph. A damaged component is isolated as a unit while
disjoint valid files remain admitted; no filename-ordered winner, repair, or
partial run is selected.
Exact selected capture is a distinct source variant used by
ptc transcript RUN_ID and by one through sixteen repeated --run flags on
private-run-analysis-v2: it resolves only
<run-ref>.jsonl or <run-ref>.private.jsonl plus
<run-ref>.ptcins, never lists the granted directory, and does not
count unrelated members toward max_directory_entries, max_files, or the
aggregate source-byte ceiling. Selected snapshot identity commits to the
canonically sorted run-reference set, trace source classes, exact evidence
digests, and correlated trace IDs. The repeated-flag cohort is admitted into
one shared all-or-nothing trace and inspection snapshot; the transcript keeps
its singular path. Explicit single-file sources remain filename-agnostic and
may contain multiple complete disjoint runs; they never combine with siblings.
Private catalog generation and selected-set binding
private-run-catalog-v1 is the discovery authority, not a lightweight form of
admission. One owner captures a generation once: each trace candidate
contributes at most its first 65,536 bytes and last 65,536 bytes, and each
.ptcins candidate contributes only the fixed 16-byte header and 192-byte
footer. The probe never decodes an inspection frame or an intervening trace
event. Before any read it compares lstat inventory identity with the opened
descriptor, and after the read it checks the path again. Replacement or drift
therefore isolates the row instead of describing bytes from another file.
The generation commitment is domain-separated by ptc-run-catalog-v1 and
covers excluded_files plus every sorted run stem and both halves' observed
state, descriptor identity, and exact probe-byte digest. Absent halves commit
to absence. The returned catalog_digest identifies that immutable captured
value; page cursors bind it and the complete filter query. Neither the digest
nor a cursor is an admission credential, persisted catalog, or input to
private-run-analysis-v2.
Catalog capture has finite ceilings: 4,096 entries per directory, 1,024 joined run stems, 2,048 encoded bytes per row, 4,194,304 retained bytes, 1,000,000 result bytes, a 5-second bounded listing operation, a 15-second capture deadline, and bounded listing/capture heaps. Entry defects are isolated in rows; unavailable roots and whole-operation bounds refuse startup. The owner holds detached metadata only and no file descriptors after capture.
Repeated --run starts a different owner and binds admission to the sorted
set of one through sixteen explicit command run references. For every member,
it resolves exactly two possible trace names and one inspection name without
listing, pins regular descriptors, validates complete embedded identities,
rehashes the selected trace bytes, confirms the inspection seal and range
digests, and proves trace-to-inspection correlation. The snapshot identity
commits to the selected set, source class, evidence digests, and correlated
trace IDs; argument order does not affect it. Unselected paths and a prior
catalog digest contribute no authority.
Trace/inspection source bytes, trace retention, inspection logical index entries and bytes, actual retained indexes, and admission heap are aggregate across the selected set. The inspection record bound remains per artifact; one pinned inspection handle exists per admitted run. Failure is all-or-nothing: partial indexes and handles are deleted before returning the path-free refusal. Normal close, construction failure, deadline, owner death, and explicit stop all tear down the catalog or selected owners and their resources; already sealed input artifacts remain untouched.
One directory holds both sanitized and private trace files, and one source
grant reads exactly one kind. A run listing and a counters query therefore
report the trace files that grant refused to read, as
excluded_private_trace_files on a sanitized source and
excluded_sanitized_trace_files on a private-only source. The field is absent
when nothing was excluded, and inspection artifacts are never counted: they are
a separate artifact class rather than withheld runs. Run-scoped answers never
carry the field. The count is advisory evidence about the directory, not part
of the source identity, so a concurrent write to the kind a grant does not read
never invalidates the evidence it does read.
The directory_admission_v1 source identity commits to the grant class; every
selected raw relative-name byte string, source class, length, and content
digest; complete filename and embedded claims and reasons; admitted events and
their per-run provenance; and untruncated isolation components. It never
commits to an absolute path, opposite-class basename or bytes, advisory
exclusion count, capture time, or presentation caps. Snapshots retain that
complete detached admission value—including the compiled analysis
projection—and charge it against the 32,000,000-byte retained ceiling.
Direct directory queries build and charge the byte-identical detached value
transiently for every call. Their cursors therefore invalidate when any
selected admitted or isolated evidence changes; snapshot cursors remain bound
to the retained value.
Every list_runs page and counters result reports grant-visible damaged
components in an isolation object, independent of filters and pagination.
Its exact component_count, source_count, and known_run_count count the
complete isolated graph; known_run_count is the distinct union of valid
filename and decoded embedded run claims. reasons follows the classifier's
fixed reason order and gives exact component and source counts for each reason,
with a multi-reason component contributing once to every applicable reason.
Run-scoped successful answers omit isolation.
The diagnostic examples list contains at most 16 components in classifier
order. Each example contains at most eight sorted, canonical UTF-8 basenames,
while source_count, sources_omitted_count, and
examples_omitted_count state the exact omissions, including invalid
basenames that cannot be displayed. Result fitting removes example source
names and then examples from the deterministic tail, updating those omission
counts, before shortening a run-item page. The exact counts and reason totals
are never removed; if those fixed fields plus the counters aggregate or an
empty page envelope do not fit, the query returns result_limit_exceeded.
Normal trace sinks sanitize before persistence. Private canonical event sinks
use the separate fail-closed policy specified by the event-sink section of the
PtcRunner.Kernel.EventSink module documentation, but retain the same event
vocabulary rather than capturing exact payloads or source.
Malformed or unsupported explicit single-file canonical events fail closed. For directory discovery, malformed JSONL, unsupported versions, filename mismatch, identity conflicts, sequence conflicts, lifecycle conflicts, non-regular members, and stably unreadable members isolate their complete connected component. Invalid raw basenames remain in immutable proof and counts but are never emitted as source metadata.
Source grants and authority
Trace access is authority-bearing and source scoped. A TraceLog capability is constructed from exactly one explicit read-only grant:
- one JSONL file;
- one directory root;
- one bounded in-memory sink.
“Available runs” means runs present in the granted source, never every trace on the host.
The implementation:
- normalizes and confines file access to the granted root;
- rejects traversal, symlink escape, and unsupported file types;
- applies one aggregate input-byte limit across directory files;
- for a selected canonical run, opens only the exact named candidates and applies the existing per-file source limits;
- exposes no ambient filesystem operations;
- keeps private canonical event-source access separate and explicit;
- never infers access from visible names, tags, or run IDs, and never treats a selected filename as authority over the embedded run identity.
The analysis prelude contains no authority. It requires host run-analysis
capabilities whose source grant provides authority. Missing requirements fail
before workflow code runs.
Trace-query capabilities belong in the mission environment when subordinate programs may inspect traces. Workflow and mission environments never inherit each other's trace grants.
Run identity
Every run has a stable run ID and trace ID. Events carry a monotonic sequence and timestamp, plus evaluation/capability IDs where applicable.
Canonical loading also validates each run's lifecycle. run-started must be
the first event for that run and may occur exactly once. A run may remain open
or end with exactly one run-stopped; no event may follow it. Histories that
start late, stop twice, or continue after stopping fail closed as
:malformed_source.
Grouping does not depend only on filenames. Duplicate events may be
deduplicated only by a documented stable identity such as {trace_id, seq}.
Every event uses schema version 2. run-started requires the plural
missions metadata map. Every event whose environment is mission requires
mission_name; workflow and lifecycle events forbid it. Unsupported versions
and missing current fields fail closed rather than being inferred or projected.
Every prelude projection on run-started — the workflow projection and each
mission's alike — is validated to producer grade rather than interpreted
leniently: unique component IDs, a dependency_indices list of the same
length as component_ids whose entries are unique and ascending and name only
strictly earlier positions, and a hash that is a bare lowercase SHA-256
digest. A nil hash means no bundle at all and can therefore name no
components; a bundle always commits to a digest, whether or not it compiled
any component. The projection
is the only canonical commitment a private prelude-source record can be
proven against, so a projection that cannot be reconstructed is
:malformed_source rather than a weaker fact.
Required run metadata
analysis/runs and analysis/open expose bounded sanitized metadata
sufficient to select a run without loading its activity:
- run ID and trace ID;
- start and stop timestamps;
- status and terminal reason when present;
- the successful terminal
result_hashwhen present, without the result value; - workflow/agent name when supplied;
- model and provider identifiers when recorded by a provider;
- total and subordinate-evaluation counts;
- workflow and mission capability-call counts;
- LLM-call summary derived from named
llm-requestevents when applicable; - the exact closed
llm_spendprojection retained byrun-stopped, so the Viewer distinguishes empty, incomplete, unpriced, available, and overflow spend without reconstructing or guessing pricing state; - error count and duration summary;
- one-way fingerprints of caller-supplied name/model/provider labels, plus finite canonical tag keys and enumerated values;
- the effective workflow prelude plus a sorted
missionsmap. Each mission entry contains its prelude component IDs, hashes, compact dependency projection, and full/model inventory hashes and byte counts; and - effective workflow and mission prelude component IDs, hashes, and the
compact dependency projection (
dependency_indices, positionally aligned withcomponent_ids; every entry lists unique ascending indices strictly earlier than its own position); - the run-started event's positive sequence in
positions, so start-derived provenance such as component overrides and prelude identities is directly citable without a second turn query. Outcome, error, and aggregate-count claims still cite the canonical turns that support them; - the bounded
component_overridesrecorded at run start, including component, base-source, and effective-source identities, so run discovery exposes treatment assignment rather than requiring one provenance query per run; - safe connector snapshots containing public names, effects, schema hashes, and snapshot hashes, plus an exact resolved LLM model only when the active model adapter attested it as public; no private target, endpoint, or session data is included;
- an optional server-owned session-profile ID and SHA-256 authority digest;
- completeness and truncation indicators;
- schema version;
- whether the source is sanitized normal data or an explicitly granted private source.
Normal metadata excludes credentials, headers, environment values, callback
terms, arbitrary provider metadata, exact private prelude source, and unbounded
messages/tool results. Absent optional facts remain absent or nil.
Query contract
The shipped analysis prelude exposes a small navigation namespace:
(analysis/runs options)
(analysis/open run-id)
(analysis/read run-id options)
(analysis/counters filters)Examples:
;; Either profile.
(analysis/runs {"status" "error" "tags" {"stage" "failed"} "limit" 20})
(analysis/open "run-id")
(analysis/read "run-id" {"collection" "activity" "limit" 100})
(analysis/counters {"tags" {"cohort" "candidate"}
"from" "2026-08-01T00:00:00Z"
"to" "2026-09-01T00:00:00Z"})
;; private-run-analysis-v2 only: every collection but `activity` is private
;; authority, and returns an :invalid_request envelope under the public profile.
(analysis/read "run-id" {"collection" "turns" "limit" 20})
(analysis/read "run-id" {"collection" "generated_sources"
"prelude_call" "workspace/read"})The internal canonical turn and counter readers accept mission_name. Run filters
select runs first, then mission matching retains only events attributed to that
mission. Counters—including event/error/run counts, evaluations,
evaluations_by_mission, capability calls, and LLM usage—are derived from that
narrowed event set. Workflow and lifecycle events never match a mission filter.
analysis/runs
Discovers runs in the granted source. Filters are limited to run/trace ID, status, bounded exact-match tags, workflow/agent name, workflow bundle hash, model/provider when present, timestamp range, limit, and cursor.
The default view is "summary" and projects each item to run_id, status,
duration_ms, llm_calls, evaluations, terminal_reason, complete, and
truncated. Set "view" to "full" for the complete sanitized metadata
record described above. Pagination and filtering are applied before this
presentation projection, so the cursor and selected run set are identical in
both views.
Default ordering is deterministic: newest start timestamp first, with run ID as a stable tie-breaker.
analysis/open
Returns metadata for one source-visible run or a uniform not-found/denied error.
It does not return all evidence implicitly. Its collections catalog names
each collection, its authority, availability, exact filters, and stable order,
so a caller can discover the next read without knowing the storage schema.
Every descriptor also names snapshot_domain, sequence_domain, and an
identifier_locations map. canonical_trace sequences belong only to the
canonical run event stream. private_inspection sequences belong only to one
inspection artifact. The turns descriptor instead reports
sequence_domain: "reconstructed_stream": {stream_id, turn} is its identity,
while its request/response sequence fields remain in the private-inspection domain.
Identifier locations use dotted paths, with [] marking values nested in a
list, such as generated[].evaluation_id on a turn.
The model_exchanges and capability_calls entries additionally advertise
item_completeness_field: "complete?"; collections without per-item
completion semantics omit that catalog field.
Every collection the private inspection record counts also carries
item_count, the number of items that run retained for it. A zero is a
measured zero for that run, not an unread or withheld page. activity is
canonical rather than retained, so it is the only collection that owns no
retained count and omits the field, as does every collection on a capture
with no private inspection record.
The turns entry also owns the transcript projection boundary:
projection_scope: "model_conversation" and projection_not_included name
the scope certified by a published transcript and the evidence surfaces it
excludes. Transcript publication derives its header from these catalog fields.
analysis/read
Returns one native bounded page from the named collection. Every page has
items, next_cursor, truncated, omitted_count, and snapshot_hash;
callers follow next_cursor explicitly. The wrapper does not aggregate pages,
diagnose failures, or hide primitive pagination. limit is an item-count upper
bound, not a promise: the query may return fewer items so the complete page,
including snapshot metadata, fits both its encoded and retained-size ceilings.
activity is public canonical activity in ascending sequence. Private captures
also advertise turns, model_exchanges, capability_calls,
provider_exchanges, generated_sources, prelude_sources,
execution_prints, and execution_errors. Public recipes report those private
collections as unavailable and reject reads without changing authority.
provider_exchanges holds MCP wire traffic and nothing else. Only the MCP
client writes its records, so a run that installed no MCP provider retains no
provider exchange and advertises item_count: 0. No LLM wire bytes are
retained on any surface: model_exchanges holds the normalized request the
runtime handed the model capability and the value that capability returned,
not the bytes exchanged with the provider. Provider-side content that never
reaches that boundary — billed reasoning tokens with no returned text, for
instance — therefore leaves no content anywhere. Attested usage is the only
record that it happened: the tokens map a normalized model_exchanges
result carries when the provider reported usage, and the analysis/counters
aggregate, which counts missing_usage_calls separately rather than reading
absent usage as zero.
Every raw model_exchanges and capability_calls item carries complete?.
Completed items retain their result, output sequence, and output timestamp.
An interrupted input-only attempt has complete?: false and omits those three
terminal fields. When a callback raised under inspection authority, the item
also carries an exception object plus exception_sequence and
exception_timestamp; these fields remain present if interruption occurred
after the exception evidence was retained but before the normalized output was
retained. Items without such evidence omit the fields rather than setting them
to null. The run metadata counts all admitted attempts and separately reports
capability_exceptions, incomplete_model_exchanges, and
incomplete_capability_calls, including zero on complete runs.
turns is the only compiled convenience collection. Each item is one model
turn with the newly added messages, response, matching generated programs, and
stable stream/turn identity. Turn numbers start at one independently within
each reconstructed stream; {stream_id, turn} is the identity, while
request_sequence only orders records inside the inspection snapshot. It must
not be compared with canonical activity sequence numbers. A turn carries the
system prompt that shaped it. Each returned page is compacted on its own,
after filtering and pagination, so every stream present in a page starts with
its effective prompt and an unchanged prompt is not repeated on every turn.
An elided system therefore means "unchanged since the last turn in this page
that carried one", never "none was sent": stream linkage keys on the request
messages alone, so one stream can span turns whose evaluated program supplied
different prompts. Compaction is idempotent, so a presented conversation
re-compacts the pages it collected without losing a prompt at a page seam.
Exact raw model requests remain available through model_exchanges. Page-level
evidence reports canonical completeness, missing exchanges, and ambiguity
separately from pagination. Interrupted model inputs remain raw evidence but
are excluded from turns, so the canonical missing-exchange count records the
gap instead of a fabricated assistant response.
Generated programs carry prelude_calls_available? and a sorted
prelude_calls list of {ref, component_id} entries. Exact
prelude_call/prelude_component filters work on both generated_sources and
turns. The association between a turn and generated source is explicitly
source_match. Matching partitions retained tool-call occurrences and
generated-source records by source. A source can match only an earlier model
response, and an association is exact only when every complete chronology-
respecting one-to-one assignment puts that evaluation on the same turn.
Repeated identical programs are therefore distinct occurrences;
multiple identical calls in one response retain multiple evaluation records on
that turn. Delayed evaluation may cross later model requests without losing its
candidate producer. If the retained occurrence counts differ, or multiple
complete assignments put an evaluation on different turns, every still-
possible evaluation retains one chronology-respecting match marked ambiguous,
rather than being multiplied across candidate turns or given a fabricated
causal identity.
Private execution errors, generated programs, and effective prelude sources may
carry a relationships list. Each relationship has this closed shape:
{
"rel": "direct_boundary_producer",
"semantics": "causation",
"target_collection": "activity",
"filters": {"evaluation_id": "mission-evaluation-id", "status": "returned"},
"state": "complete"
}The closed relation IDs are boundary_failure, child_evaluations,
dependency_prelude_source, direct_boundary_producer, generated_source,
producing_turn, and referenced_prelude_source.
target_collection and non-null filters are the exact options to add to the
same run's next analysis/read; callers must not follow a null filter. State is
one of complete, incomplete, ambiguous, or unavailable. Ambiguous
producer candidates are emitted as separately followable exact filters, each
marked ambiguous, rather than as an unbounded scan.
Semantics are intentionally narrower than the relation names. causation
requires a workflow boundary failure or a direct-return origin marker plus
exact equality with the retained successful kernel-eval result. A direct
producer is complete only when the canonical trace proves that the workflow
evaluation stopped with error and that the named mission evaluation is
parented by it and stopped with returned or continued; the exact activity
filters retain those runtime statuses. A canonically successful workflow makes
both causal relations unavailable. nesting is reserved for the validated canonical
parent_evaluation_id edge.
association covers source identity, static prelude references, and the
generated-source/turn source match. A source match can be ambiguous; a parent
edge alone is never promoted to causation. Truncated producer evidence is
incomplete, a complete search with no match is unavailable, and no relation
compares canonical and inspection sequence values.
dependency is reserved for the frozen prelude graph. Each effective prelude
source carries one dependency_prelude_source relation per direct dependency
recorded at run start. A component ID alone is not an occurrence identity — the
same component can be frozen into the workflow environment and into several
missions with different dependency edges — so every filter set repeats the
owning environment and, for a mission occurrence, its mission_name. The
edges are read only from a graph that satisfies the complete positional
contract: dependency_indices aligned with unique component_ids, and each
entry holding unique ascending indices strictly earlier than its own position.
A graph that is absent, or that violates any part of that contract, yields one
incomplete relation with null filters rather than a guessed edge. A component
with no dependencies yields an empty list, which is a different claim from
incomplete.
Generated entries embedded in turns carry the same relationships list as
the matching generated_sources item, so a walker that starts from a turn can
follow evidence without an extra exact read to obtain the links.
Provider response usage omits total_cost when pricing is unavailable. A
present zero-microunit USD object is therefore a measured zero-cost response,
not an unknown cost. Decimal provider values are parsed exactly and rounded
upward to one microunit; floats never remain accounting authority.
The canonical LLM usage summary is shared by trace counters and the V4 run
envelope's execution.usage. For routed llm-request calls, llm_usage groups
stopped events by model alias and installation revision. Each row reports total
and successful calls, calls with valid usage, calls that may have dispatched
but lack usage,
and sums of the closed input, output, cache_creation, cache_read, and
total_cost fields. Every row has usage_overflow; sums saturate at
9_007_199_254_740_991, and a true flag means at least one displayed value is
only a lower bound. A row includes aggregate total_cost only when every call
that could have reached a provider has valid usage that reports cost; otherwise cost is unknown
and omitted. A revision change creates a separate row rather than silently
combining unlike deployments.
llm_usage_by_model groups the same eligible stopped calls by the exact
adapter-attested resolved_model stored in that run's LLM provider snapshot.
Rows use the same call, success, usage-presence, and closed usage-sum fields as
llm_usage, replacing alias and revision with resolved_model, and sort by that
string. Equal model targets combine across aliases and installation revisions;
different targets remain separate even when an operator reused one revision.
Correlation is run-scoped through {run_id, alias, installation_revision} and
accepts exactly one current LLM snapshot whose closed shape and two identity
hashes verify. Missing, private, legacy, malformed, hash-inconsistent, replay,
custom, or duplicated mappings do not fail the query. Their otherwise
llm_usage-eligible calls increment unattributed_model_calls. Consequently:
sum(llm_usage_by_model[*].calls) + unattributed_model_calls
== sum(llm_usage[*].calls)Snapshot lookup uses all events from the run-filter-selected runs before a
mission filter narrows counted calls. Capability events continue to carry only
alias/revision routing identity; they do not duplicate model identity. The
additional rows remain subject to the existing aggregate result-byte limit.
Every finished V4 run envelope also publishes the sealed run-state
llm_budget and llm_spend values that canonical run-stopped.data.usage
retains. Every current run-stopped event must carry data.usage with a valid
terminal llm_budget; omission makes the trace malformed. llm_budget is
authored only by RunState rather than reconstructed
from capability events. Its exact total_tokens and cost ledgers are null
when disabled; enabled ledgers expose their limit, zero terminal reservation,
charged and remaining capacity, refusal count, and
available | incomplete | overrun state. Overrun clamps remaining to zero.
The Viewer run catalog consumes this same field and validates the terminal
shape before displaying it.
llm_spend has the closed states empty, incomplete, unpriced,
available, and overflow.
empty, incomplete, and overflow contain only state; the latter two
ordinary states require complete non-negative input and output totals, and only
available requires the canonical USD/microunit total_cost. Any aggregate
token or cost sum beyond the canonical maximum produces overflow. Consequently an
unknown cost never deserializes as measured zero. Command projection validates
this value through LLMUsageSummary; absence or malformed shape invalidates the
command outcome rather than triggering reconstruction from trace rows. The
Viewer run catalog consumes this same terminal field and does not maintain a
second totals vocabulary. A completed LLM error that may have dispatched and
has no usage makes spend incomplete; a provider error carrying trusted
usage contributes the same observed tokens and cost as a success while leaving
successful_calls unchanged. Trusted not_dispatched provenance is excluded
because no provider request could have been billed.
The command envelope additionally publishes llm_usage_state. Terminal
accounting pairs llm-request start and stop events by capability_id; an
unmatched start is an observed call with unknown usage (missing_usage_calls
increments even when successful_calls does not). Dropped
capability-started or capability-stopped events, or a malformed pairing,
produce "unavailable" with null aggregate fields. A validated run with no
calls produces "available", empty arrays, and zero unattributed calls.
analysis/counters
Returns one canonical trace aggregate for the selected run cohort. Filters are
the existing counter keys: status, run_id, trace_id, tags, name,
bundle, model, provider, from, to, and mission_name. There is no
limit, cursor, view, run_ids, or resolved-model filter. Unknown keys
fail at the capability schema before dispatch. Tags are limited to 16
properties and 256-character property names at that same schema boundary.
UTF-8 byte ceilings for keys and values remain TraceLog's semantic
invalid_query. model keeps its run-filter meaning and is not reinterpreted
as adapter-attested resolved_model.
The result is the captured TraceSnapshot :counters map, including
events, runs, errors, evaluations, evaluations_by_mission,
workflow_capability_calls, mission_capability_calls, llm_usage,
llm_usage_by_model, unattributed_model_calls, and snapshot/source metadata.
If the complete aggregate exceeds the snapshot result ceiling, the call fails
as result_limit_exceeded rather than returning a truncated map.
A filter-defined cohort is one counters call. An explicit list of selected run
IDs is one call per run_id, with the caller reducing the returned
llm_usage_by_model rows in PTC-Lisp. Those rows carry calls,
successful_calls, usage_calls, missing_usage_calls, usage_overflow, and
a nested usage map with input, output, and optional fixed-point
total_cost. Group by
resolved_model and sum the call counters plus nested token keys. The run-ID
list is cohort selection data; each result already carries attested attribution
or an honest unattributed count. Absent total_cost is unknown spend, not
zero. Costs add through their microunits fields, never by adding the objects
themselves. Preserve usage_overflow, check sums against
9_007_199_254_740_991, and omit cost whenever any contributing row withholds
it.
(defn withheld-cost? [rows]
(some #(not (contains? (get % "usage") "total_cost")) rows))
(defn cost-microunits [row]
(get-in row ["usage" "total_cost" "microunits"]))
(def max-usage 9007199254740991)
(def token-keys ["input" "output" "cache_creation" "cache_read"])
(defn reduce-model-rows [rows]
(->> rows
(group-by #(get % "resolved_model"))
(map (fn [[model group]]
(let [withheld? (withheld-cost? group)
keys (filter (fn [key] (some #(contains? (get % "usage") key) group))
token-keys)
sums (into {} (map (fn [key]
[key (reduce + (map #(get (get % "usage") key 0) group))])
keys))
cost (if withheld? 0 (reduce + (map cost-microunits group)))
overflow? (or (some #(get % "usage_overflow") group)
(some #(> % max-usage) (vals sums))
(and (not withheld?) (> cost max-usage)))
usage (into {} (map (fn [[key value]] [key (min value max-usage)]) sums))
usage (if withheld?
usage
(assoc usage "total_cost"
{"currency" "USD" "microunits" (min cost max-usage)}))]
{"resolved_model" model
"calls" (reduce + (map #(get % "calls") group))
"successful_calls" (reduce + (map #(get % "successful_calls") group))
"usage_calls" (reduce + (map #(get % "usage_calls") group))
"missing_usage_calls" (reduce + (map #(get % "missing_usage_calls") group))
"usage_overflow" (boolean overflow?)
"usage" usage})))))
(def selected ["run-a" "run-b" "run-c"])
(def pages (map #(analysis/counters {"run_id" %}) selected))
(def model-rows (mapcat #(get % "llm_usage_by_model") pages))
(def unattributed (apply + (map #(get % "unattributed_model_calls") pages)))
{:models (reduce-model-rows model-rows) :unattributed unattributed}Pagination, ordering, and bounds
Every collection query has:
- a conservative default and hard maximum limit;
- a deterministic opaque cursor;
- deterministic ordering and tie-breakers;
- maximum encoded and retained result sizes under one result ceiling;
- truncation and omitted-count metadata, which report pagination only and never source-kind exclusion;
- one aggregate source-read byte cap;
- bounded filter/tag/name lengths and counts.
A cursor is bound to the source identity, query operation, normalized filters,
ordering, and page limit that produced it. Changing any of them or reusing a
cursor for another operation fails as an invalid query; changing the source
fails as source-changed.
If a page would exceed either result-size measurement, return the largest valid prefix plus explicit truncation/next-cursor metadata, or a stable bounded error when even one item does not fit. Sizing covers the final page shape, including snapshot metadata. Never build an unbounded result and truncate only after allocation.
Preview fields are bounded independently. Arguments/results, messages, prints, memory diffs, and program source follow their own projection ceilings.
Capabilities and swappable preludes
Analysis capabilities follow the standard Kernel envelope and are named
analysis-counters, analysis-open, analysis-read, and analysis-runs.
All four delegate to PtcRunner.Kernel.RunAnalysis. The analysis prelude is a
thin cap/unwrap! layer; Viewer, CLI, and embedders call the same read model.
Capability error envelopes fail evaluation rather than flowing into projections
as ordinary empty data.
Preludes may change ergonomics, projections, defaults, or analysis policy. They cannot expand the source grant, bypass bounds/sanitization, or acquire private trace access.
Events and private data
The generic Kernel emits lifecycle, subordinate-evaluation, capability, resource, annotation, and terminal facts. Providers may attach safe typed metadata without making Kernel understand provider-specific transcripts.
A successful run-stopped event includes data.result_hash: sha256: plus
the lowercase SHA-256 digest of the successful result value's deterministic
canonical JSON bytes. ResultArtifact writes those same bytes, so a trusted
host can bind an artifact to the run that produced it without exposing the
result in the public trace. Failed runs omit the field.
A shipped agent.core loop whose caller propagates exhaustion records
failure_kind: "turn-limit", limit: "agent_turns", the validated
limit_value from 1 through 128, and limit_reason on the failed
run-stopped event. limit_reason is one of turn_limit_exceeded,
intermediate_result, evaluation_error, or protocol_error, naming what the
final turn produced: a model still working, a program that failed, or no usable
tool call at all. A reason outside that set is dropped rather than recorded.
Other recognized explicit failures retain only their bounded failure taxonomy;
these fields never admit caller-supplied prose.
A capability-stopped event with status: "error" carries the closed Lisp
envelope kind and, when present, reason — the same payload-free class
execution_errors already expose. An unrecognized envelope atom is retained
only as a one-way fingerprint (kind_fingerprint / reason_fingerprint).
Arguments, results, details, and messages stay off this event. Successful
stops omit these fields.
A successful routed llm-request stop carries the router-owned installation
alias and installation_revision, plus normalized usage when supplied. For
non-streaming tool responses it may also carry the normalized finish_reason.
When that reason is length and the adapter can authenticate the cap after
provider request normalization, output_limit records name: "max_tokens",
the positive effective request value, and the canonically ordered binding
list. This is the cap PtcRunner sent, not an assertion about a provider's
private ceiling. A rewritten, removed, or ambiguously resolved cap is omitted.
For live Kernel requests, the closed binding vocabulary distinguishes the
effective application llm_request_output_tokens ceiling (application_limit)
from installation params.max_tokens (installation_param) and retains both
when they tie. The application value may be the manifest request or its
compiled default, each capped by a lower installed host ceiling. This provenance
is sealed during model preparation rather than
reconstructed from the effective value after the request.
When proven truncation prevents a usable shipped-agent action, the failed
run-stopped records reason: "model_output_truncated" and the authenticated
alias. It also records limit: "max_tokens", limit_value, and
limit_bindings when that provenance is available. Private
capability-output records contain the same normalized response and alias; the
private execution-error contains only the authenticated alias plus any
available bounded cap details. Raw provider responses and raw stop reasons are
not retained.
The implemented private event policy stores the same canonical event
vocabulary as the normal policy. It changes sink failure behavior, file
permissions, and discovery; it does not capture prompts, responses, capability
payloads, generated programs, or prelude source. Private JSONL destinations are
restricted to owner read/write permissions before any event payload is
appended. They use the reserved .private.jsonl suffix, and normal
file/directory sources and Viewer discovery reject or omit that suffix.
Implemented 0.x developer-inspection contract
Sanitized subordinate evaluation-started data adds:
parent_evaluation_id— the enclosing workflow evaluation ID when the subordinate evaluation was launched through the workflow'skernel-evalroute; the matchingevaluation-stoppedevent repeats this ID;source_hash— lower-case SHA-256 hex over the exact UTF-8 source bytes passed toLisp.run_native/2;source_bytes—byte_size(source)over those same bytes; andprogram_kind: "ptc-lisp"alongside the existingenvironment: "mission"andevaluation_id.
It must not contain exact source. The activity collection returns canonical event
data, so embedding source in a supposedly private event would collapse source
authorization into the ordinary activity query.
Private generated_sources copy parent_evaluation_id only from that
canonical start event, and reconstructed turns can be filtered by either the
child evaluation_id or its parent_evaluation_id. Thus an execution error's
workflow evaluation ID can select its child programs and turns without
comparing canonical and inspection sequence numbers. The edge proves nesting,
not that a particular child caused the workflow error.
Canonical loading validates the edge before exposing it: the parent must be a preceding, still-open workflow evaluation in the same run, only a mission evaluation may name it, and a matching child stop must repeat the same parent. Orphaned, cyclic, wrong-environment, or contradictory edges make the source malformed rather than becoming navigation evidence.
Optional sensitive development capture uses a separate private inspection record stream, not a canonical event. Every V1 evidence-frame payload is one deterministically encoded JSON object with this exact envelope:
{
"schema_version": 10,
"run_id": "run-id",
"trace_id": "trace-id",
"sequence": 1,
"timestamp": "2026-07-16T12:00:00.000000Z",
"record_type": "capability-input",
"correlation": {"capability_id": "capability-id"},
"payload": {}
}Keys are exact. sequence is positive and strictly increasing within the
artifact. The timestamp is UTC ISO 8601. run-result correlates to nothing
and carries an empty map. Every other record carries exactly one of
capability_id, evaluation_id, or component_id, and the three MCP record
types pair their capability_id with the positive integer request_id that
identifies the exchange. Capability and evaluation values must occur in the
canonical trace for the same run unless that trace explicitly proves the
corresponding start events were dropped. A component value must occur in the
canonical run-started prelude component IDs for the record's environment.
The current record types and payloads are:
| Record type | Correlation | Exact payload fields |
|---|---|---|
capability-input | capability_id | environment, name, arguments; llm-request also carries request_hash |
capability-exception | capability_id | environment, name, exception_class, message, message_truncated, stacktrace, stacktrace_truncated |
capability-output | capability_id | environment, name, result or result_identity |
evaluation-source | evaluation_id | environment, program_kind, source, source_hash, source_bytes |
evaluation-analysis | evaluation_id | environment, mission_name, prelude_calls |
prelude-source | component_id | environment, source, source_hash, source_bytes |
mcp-request | capability_id, request_id | transport, body |
mcp-response | capability_id, request_id | transport, body or body_identity |
mcp-stderr | capability_id, request_id | transport, text, truncated |
execution-prints | evaluation_id | environment, prints, truncated |
execution-error | evaluation_id | environment, kind, reason, details |
explicit-failure-value | evaluation_id | environment, value |
A read tool the host declared with inspection_capture: "digest_results"
retains value identities in place of two of those payload values:
capability-output carries result_identity instead of result, and
mcp-response carries body_identity instead of body. An identity is the
exact object {"encoding": "ptc-deterministic-json-v1", "sha256": "sha256:...", "encoded_bytes": N}, where the digest covers
"ptc.inspection-value.v1\0" || u64(encoded_bytes) || encoded_value over the
value normalized as a retained record would have normalized it. encoded_bytes
is the size of that deterministic JSON encoding, not the MCP wire size.
The alternatives are per record, not per artifact: record types, ordering,
correlation, joins, and counts are identical to a fully captured run. The
digest decision uses only what the transport has already computed: MCP error
and isError bodies, bodies rejected as oversized or malformed, capability
error envelopes, capability-exception, and mcp-stderr stay full. A value
the transport accepted and a later stage rejected -- tool-result
normalization, retained-size admission, bounded output validation -- keeps its
full error envelope while the wire body remains an identity; the mapping is a
read, so a full-capture rerun recovers the value and its identity proves it is
the same one. The sink computes the identity and the decision adds no work, so
digest capture cannot change an emitting process's heap behaviour. An identity
attests that a specific normalized value crossed the boundary at that record's
position in the observed order. It does not retain the value, prove derivation
of later values, or attest the original MCP wire bytes.
Sealed inspection artifact V1
The private inspection artifact suffix is .ptcins. It is a sealed binary
container, not JSONL. All multibyte integers are unsigned big-endian and all
offsets are absolute:
16-byte header
"PTCINS01" | format_version:u16 | schema_version:u16 | header_size:u32
zero or more evidence frames
payload_bytes:u64 | deterministic_json_payload:payload_bytes
192-byte footer
"PTCIFTR1" | format_version:u16 | schema_version:u16 | footer_size:u32
total_bytes:u64 | evidence_offset:u64 | evidence_bytes:u64
record_count:u64 | first_sequence:u64 | last_sequence:u64
run_id_sha256:32 | trace_id_sha256:32 | evidence_sha256:32
artifact_digest:32V1 uses format version 1, the installed inspection schema version, header size
16, and footer size 192. Evidence starts immediately after the header and the
footer immediately follows it. A zero-frame artifact uses zero for
record_count, first_sequence, and last_sequence; its footer identity
hashes are resolved independently against the paired trace. Gaps, overlap,
trailing bytes, nonzero reserved representations, unknown versions, integer
overflow, truncation, or inconsistent offsets and counts are malformed.
evidence_sha256 covers the exact length prefixes and payload bytes. The
artifact digest covers the exact header and framed evidence followed by the
exact footer with its final 32-byte digest field zeroed. Run and trace identity
fields are SHA-256 of their UTF-8 bytes. Payloads use the envelope key order
above and DeterministicJSON UTF-8 byte ordering for nested object keys.
Admission rejects duplicate JSON keys, non-canonical or invalid record shapes,
and any footer claim it cannot reconstruct independently.
Production reserves an absent final destination before execution, writes the header and each accepted frame through owner-atomic offset-returning appends, updates counts and digests incrementally, appends and synchronizes the footer, then publishes the staging inode without replacing an existing destination. The sink retains no complete record list and publication never constructs a complete artifact binary. Refusal or interruption before the synchronized footer leaves no published partial artifact; a destination collision at publication fails closed.
Reader admission inventories only regular files, then verifies the opened descriptor's device, inode, type, and size against that inventory before pinning it; a pathname replacement or symlink substitution fails closed. One bounded decode scan validates every frame, reconstructs collection order, filter postings, joins, turns, counts, results, conversations, and relationships, and checks those facts against the independently admitted paired trace. Derived locator and projection metadata is inserted into private, owner-bound ETS tables under entry, logical-state, actual retained-state, heap, and deadline ceilings. A second pass rehashes the exact evidence bytes without decoding records and confirms the footer, identities, counts, and opened source remained stable. A snapshot capability is returned only after all artifacts pass; any refusal tears down every table and pinned handle while leaving already published evidence durable.
Queries select locators through ETS and read only the pinned ranges needed for
returned records and their join or turn dependencies. Every such range is
digest-checked against admission before a page is returned. A required-range
change returns source_changed without partial items or a cursor. Append,
truncate, or opened-size/footer inconsistency fails the next query. Replacing
the pathname does not redirect the pinned reader, while unrelated unread
same-size corruption may remain latent.
omitted_count is the exact cardinality remaining in the admission-validated
ETS result set after the returned logical position. Frames represented only by
omitted locators are not reread merely to count them. Their same-size mutation
may remain latent until selected or needed by a returned join. Cursors bind the
admitted artifact digest, run and query identity, filters, order, page-limit
semantics, and logical position; they never require a complete artifact rehash
per page.
The finite production authorities are:
max_record_bytes: 2,000,000 encoded payload bytes;- framed evidence: 536,870,912 bytes, excluding the fixed header/footer;
- complete artifact: 536,871,120 bytes;
max_records: default 16,384 and maintained maximum 65,536;- logical index entries: 1,000,000 and logical index bytes: 128,000,000;
- aggregate derived snapshot retention: 128,000,000 actual ETS and disjoint owner metadata, paired-trace facts, and cache bytes;
- host-facing selected-source/directory bytes: at most 536,871,120;
- host-facing directory files: at most 1,024;
- host-facing result bytes: at most 1,048,576; and
- admission/query phase envelope: 15 seconds, cleanup: 5 seconds.
The producer refuses before crossing its evidence or record limit. Admission refuses before crossing record or logical retained-state authority and again after measuring actual aggregate retention. One admitted artifact owns one pinned reader handle. ETS layout, table type/count, key shape, allocation word counts, concurrency, sorting, caching, and backend choice are implementation details and are not configurable host policy.
Named-mission ownership is explicit. Mission capability-input,
capability-exception, capability-output, evaluation-source, evaluation-analysis,
prelude-source, mcp-request, mcp-response, and mcp-stderr payloads require
mission_name; workflow payloads forbid it.
Prelude uniqueness is (environment, mission_name, component_id), so the same
component ID can be inspected independently in multiple missions. Every
mission-owned query result preserves mission_name.
V8 retains the successful terminal-result record introduced in V6, at most one
per run, and adds the dedicated explicit-failure-value record:
| Record type | Correlation | Exact payload fields |
|---|---|---|
run-result | empty map | result_hash, value |
The value must already be strict JSON: string-keyed maps, lists, strings,
finite numbers, booleans, or null. Its lowercase sha256: hash is computed
from deterministic canonical JSON and must equal both payload.result_hash
and the one successful canonical run-stopped.data.result_hash. The record is
omitted for failures and successful native values that cannot be represented
as strict JSON.
For record types other than run-result, enums and map keys are normalized to
JSON strings before retention. A run-result.value is never coerced; it is
rejected unless the supplied value is already strict JSON. result is the
bounded Dispatcher envelope returned to Lisp, so llm-request input/output
records contain the provider-neutral model request and normalized response, and
MCP records contain the public connector arguments and normalized result/error.
Stdio sessions also retain bounded child-stderr text on a sibling mcp-stderr
record correlated to the same {capability_id, request_id}; those bytes can
name host paths and exist only under explicit private-inspection authority.
Captured stdio exchanges are serialized so that record is one request's capture
window. truncated is true when the launcher hit stderr_bytes or otherwise
signaled overflow.
When a capability callback raises, capability-exception retains the exception
module name, its message at no more than 4,096 valid UTF-8 bytes, and a formatted
stacktrace at no more than 64 frames and 65,536 valid UTF-8 bytes. Independent
flags say whether either string was truncated or unavailable. Formatting does
not generically inspect or retain the raw exception struct or callback
arguments. The exception-defined message formatter does receive that struct,
however, and its authored text plus source paths in a stacktrace can expose
embedded payloads, credentials, or other sensitive data that cannot be reliably
redacted. This record therefore exists only under explicit private-inspection
authority. Exit, throw, timeout, provider-process death, and inspection-disabled
runs retain no exception record. The normalized capability-output and
canonical capability-stopped event remain the existing closed
provider_error / exception shape when private retention succeeds. Failure to
retain the required exception record follows the existing fail-closed
inspection contract and replaces that outcome with inspection_sink_error.
evaluation-source is emitted only for subordinate mission evaluation. Its
hash and byte count must equal the corresponding canonical
evaluation-started fields. After successful static analysis of that source,
evaluation-analysis records the sorted unique public prelude functions the
resolved program calls as exact {"ref", "component_id"} objects. Direct
calls and callable function references count; constants do not. Parse or
analysis failure leaves the source without an analysis record, which projects
as prelude_calls_available?: false rather than an empty successful result.
The analysis record is accepted before continuation state commits; a sink or
component-mapping failure fails the evaluation without committing that
continuation. prelude-source records carry the exact
effective source of every frozen workflow and mission component, one record
per component in frozen order, emitted by the manifest-backed builder before
execution begins. execution-prints is emitted for the top-level workflow
evaluation whenever it produces println output, whether the evaluation
succeeds or fails: prints is the run's bounded println output. Public
evaluation prints are projected in one pass under both a 128-entry ceiling and
a 65,536-byte encoded JSON-array ceiling, and the truncation flag is
authoritative even when the omitted entries are empty strings; this record uses
the same projection. execution-error is emitted only when the top-level
workflow evaluation fails with a non-empty details map, where details is
the Kernel Error.details map computed for that failure. Their environment
is always "workflow", and their evaluation_id must match a canonical
evaluation-started event with environment: "workflow" for the same run.
When the workflow's return expression is directly a kernel-eval, private
details also contain bounded boundary_producer evidence: the child
evaluation_id when that retained successful result is exactly the workflow
boundary value, plus complete? indicating whether that ledger result was
intact. A syntactically transformed or independently recomputed equal value is
not treated as provenance. An empty complete list means the direct result did
not expose a valid child identity; a false completion flag forbids that
conclusion. The value itself is not duplicated in this metadata.
The input record is accepted before the callback starts. A raised callback's
exception record is accepted after the bounded provider worker has stopped and
before its normalized output is accepted. A subordinate
evaluation-started event is attempted before its source record is accepted,
and the source record is accepted before Lisp execution starts. The output
record is accepted after normalization and before the canonical stop event. A
missing output therefore means the attempt was interrupted; the loader does
not synthesize one. Failure to accept a required input/source record prevents
execution. Failure after an external read has completed fails the run but
cannot retroactively undo that read.
Artifact admission rejects ambiguous joins before publishing a snapshot or
Viewer capability. There may be at most one input, exception, and output for a capability ID, one
source and one subsequent analysis for an evaluation ID, and one source for an
environment/component pair.
A capability exception requires an earlier matching input, must precede any
output, and must repeat the exact environment, mission, and public capability
name. A following output must be the closed non-retryable
provider_error / exception envelope. An input, or input plus exception,
without an output is valid only as an interrupted retained prefix. Private
queries retain that input as complete?: false and do not synthesize terminal
fields. MCP provider
exchange projections still require a validated request/response pair; the
inspection sink retains each pair atomically so interruption cannot leave a
request without its response. Private
capability name and environment must match the canonical capability-started
event carrying the same ID, and canonical capability-start IDs must themselves
be unique. Browser indexing also refuses to overwrite a prior identity
defensively, but server validation is authoritative. There may likewise be at
most one
execution-prints and one execution-error for a given evaluation_id; a run
with no println output and no execution-phase failure emits neither.
A normal trace can prove a missing correlation only through both of its
terminal loss records: the one events-dropped.data.counts map and
run-stopped.data.usage.events_dropped must agree, and the count for
capability-started or evaluation-started must cover every distinct missing
ID of that kind. Missing evaluation IDs also retain their expected environment;
one dropped event cannot cover conflicting workflow and mission records. The
runtime emits each correlated private record only after attempting its
corresponding start event, so this typed count is authoritative evidence of
trace loss. The events-dropped marker and run-stopped must be the final two
events in that order, matching atomic EventSink finalization. The canonical
marker flags the paired inspection artifact as partial. Missing proof,
insufficient counts, duplicate canonical IDs, or an existing event whose
correlation fields disagree all fail closed.
The host enables inspection capture independently of manifest event policy and selects a fixed exact destination; a manifest or PTC-Lisp program cannot enable it or choose where it goes. The destination is restricted to owner read/write before content is written. Capture is either disabled or required and fail-closed: the inspection sink never silently drops its own records. A trace-budget loss is different — the canonical trace marks it explicitly, and a fully retained inspection stream may still be persisted as the trace-marked partial correlation overlay described above. Retention belongs to the host in 0.x.
These authorities are internal to the installed producer and snapshot. A host may lower its exposed source, file, and result ceilings, but a manifest cannot raise or select any of them. Encoded expansion under JSON escaping is checked before the frame append. Publication is atomic and no-clobber, and one artifact contains exactly one run; production never appends to or merges a published artifact.
This increment captures the normalized LLM request and response, exact
generated subordinate PTC-Lisp, exact effective prelude component source, and
connector capability arguments and normalized results/errors needed to
inspect a development run. It does not add transport headers, connector
credentials, session IDs, endpoints, arbitrary host terms, or the composed
workflow entry expression. Application arguments/results and prelude source
may themselves be sensitive, so the entire artifact is private. The exact
mission context that the model received is already part of the captured LLM
request. This is the exact provider-neutral llm-request input, not a promise
of byte-for-byte provider wire capture after adapter transformation.
The Viewer defines overlay completeness from canonical evidence. Expected LLM
calls are canonical capability-started events named llm-request; a stopped
call joins only when both private input and output exist. Expected dispatched
calls exclude reserved Kernel runtime routes (kernel-eval,
kernel-check-source, both mission
context routes, runtime usage/remaining, capability discovery, and workflow
annotation). A run cannot claim a complete overlay when it lacks a terminal
event, reports dropped events, exceeds the Viewer page budget, or lacks an
expected join. Exactly one state-aware provenance notice distinguishes a
canonical-only trace, a complete private overlay, and incomplete or failed
inspection states.
The inspection artifact is absent from TraceLog file/directory discovery and
from every primitive TraceLog query. Normal discovery explicitly rejects or omits the
.ptcins suffix rather than accidentally parsing it as canonical
JSONL.
A host-installed inspection snapshot composes its correlated canonical trace
through the four navigation operations. analysis/open includes an eligible
immutable V8 result value and canonical result_hash; an unknown run and a
known run without an eligible result remain distinct internally. Both encoded
and retained sizes must fit the snapshot result ceiling. Possessing a private
canonical event source, local Viewer access, or the active run does not imply
inspection authority. Calls emit ordinary bounded capability facts without
copying returned payloads into the trace.
Directory-backed private analysis carries trace isolation across the correlation boundary. When an inspection footer's exact run and trace digests resolve to one isolated trace component, admission checks its record frames and complete seal in disposable indexes, commits the artifact digest to inspection snapshot identity, and discards those private rows. Other inspection artifacts continue through normal trace correlation, so a healthy run remains navigable and the trace run listing reports the isolated source and classifier reason. This is isolation, not a legacy trace reader: the rejected trace representation never enters the canonical analysis projection.
Active-run trace self-query remains unsupported. Every trace capability call adds events to the same sink, while pagination cursors are source-digest-bound; the query would mutate the source it is paging. Same-run correction retains the bounded prior program directly in provider-valid assistant/tool/result history.
Workflow annotations are host stamped and cannot forge canonical events.
They use a finite semantic vocabulary: types and keys are closed, and
enumerated values (stage, kind) are closed. A phased agent-action
may also carry mission, a bounded identifier rather than a free string.
workflow.event/annotate accepts exactly these rows. A string type that
is not in the table, or a listed type with the wrong keys or values,
returns
{"status":"error","kind":"invalid_annotation","reason":"invalid_workflow_annotation"}
to the caller and does not fail the evaluation:
annotation_type | accepted data |
|---|---|
"progress" | {"stage": started | planning | executing | validating | completed | failed} — that key and no other |
"agent-action" | {"turn": 0..127, "max_turns": 1..128, "invocation": "agent-<16 lowercase hex>", "kind": tool-call | protocol-error | provider-error | max-calls | model-output-truncated}, or that plus {"phase": 0..7, "phase_turn": 0..127, "mission": <name>} — exactly four keys or exactly seven |
agent-verification | exactly status: accepted, rejected, or unresolved; no candidate, feedback, or verifier evidence |
Keyword types and keys normalize (:phase-turn → "phase_turn"). A phased
agent-action takes all three phase keys or none. Callers provide every field
except invocation; the runtime adds that run-local opaque correlation value
before validation and emission. max_turns is the configured total ceiling for
the invocation, and turn is always strictly less than it. mission is the phase's
mission name: a lowercase letter, then up to 127 letters, digits, ., _, or
-. The vocabulary never carries detailed reasons, generated source, or model
content. A non-string annotation type is a malformed call, not this
vocabulary rejection.
Viewer and CLI sharing
ptc_viewer, CLI debugging, and the model-facing analysis/* capabilities
share the same loader, metadata derivation, filtering, ordering, pagination,
and bounded RunAnalysis navigation where practical.
The viewer may render richer presentations, but it is not a second canonical
query implementation or authority source. A development mode may additionally
read one exact host-selected inspection artifact through a separate bounded
loader. The browser cannot select a server-side path, and that loader does not
become a TraceLog operation or model capability.
The trace browser itself is unauthenticated. It binds loopback by default, and
a host that binds the wildcard address publishes whatever trace and inspection
data that instance was granted to every host that can reach the port. Live
browser controls additionally require a local page authority, and a non-loopback
reporter requires the separately configured live token. Broader shared host
authorization remains deferred until a real host product exists. Local private
inspection does not wait for that work and does not change
Kernel.TraceLog ownership.
Failure algebra
Trace queries surface three capability-envelope kinds — :not_found,
:invalid_request, and non-retryable :unavailable (plus :internal for
unexpected failures). The specific condition is carried in the bounded details
string. :unavailable reports a grant-visible :run_isolated claim with the
exact detail analysis run is isolated by damaged trace evidence.
:invalid_request covers :invalid_query, :unsupported_version,
:malformed_source, :source_limit_exceeded, :result_limit_exceeded, and
:source_changed.
Details are bounded and sanitized. Host paths are not exposed beyond safe grant-relative identifiers.
Inspection production uses :inspection_sink_error and
:inspection_persistence_failed; admitted inspection sources use the snapshot
failure algebra (:malformed_source, :source_changed,
:source_limit_exceeded, :result_limit_exceeded, correlation failures, and
selected-run mismatch).
Destination preflight adds :inspection_destination_exists,
:inspection_destination_unavailable, and :inspection_destination_unsafe,
the last naming an ancestor with an untrusted owner or mode. Errors do
not include a record payload or host path. A completed Kernel result may be
returned as bounded context with :inspection_persistence_failed, matching the
existing trace-persistence distinction; persistence failure is not rewritten
as workflow failure.
Private cohort acceptance ledger
The end-to-end test owns only the cross-stage workflow. Deterministic boundary tests remain authoritative for fault states that do not need forty full private payloads:
| Acceptance behavior | Authoritative coverage |
|---|---|
| metadata-only probes and no payload reads | run_catalog_snapshot_test.exs: “a paired cohort lists safe metadata without opening either payload” |
| growth cannot alter an open generation | run_catalog_snapshot_test.exs: “an open generation is unchanged by a run published after its capture” |
| stale and filter-bound cursors | run_catalog_snapshot_test.exs: “a cursor from another catalog generation is stale” and “cursors bind every filter but not the page limit” |
| duplicate identities and missing pairs | run_catalog_snapshot_test.exs: “two entries claiming one trace identity isolate both” and “a row reports a missing half instead of refusing the generation” |
| replacement and changed metadata | run_catalog_snapshot_test.exs: descriptor-replacement tests; selected_canonical_set_snapshot_test.exs: alternate-creation, replacement, truncation, and mismatch tests |
| malformed selected versus unselected members | run_catalog_profile_test.exs: malformed unselected isolation; selected_canonical_set_snapshot_test.exs: selected malformed/version failures |
| exact candidate and correlation failures | selected_canonical_source_test.exs, selected_canonical_snapshot_test.exs, and selected_canonical_set_snapshot_test.exs |
| aggregate bytes/index retention and per-artifact records | selected_canonical_set_snapshot_test.exs: source-byte, record, and retained-index ceiling tests |
| owner death and cleanup after partial admission | run_catalog_snapshot_test.exs: owner exit; run_catalog_profile_test.exs: close/construction cleanup; selected_canonical_set_snapshot_test.exs: owner-death and partial-index cleanup tests |
| paged forty-run discovery and two independent 16-run sessions | ptc_repl_test.exs: “catalog discovery pages forty safe rows into two independent selected sessions” |
| stable closed frontend diagnostic projection | profile_diagnostic_catalog_test.exs and the selected-run JSONL assertions in ptc_repl_test.exs |
Non-goals
- ambient search over all host traces;
- arbitrary filesystem browsing;
- write/update/delete through
analysis/*; - a mutable authoritative run database;
- unbounded full-text search or arbitrary query expressions;
- automatic access to the current run's private events or source records;
- live prelude mutation or trace rewriting;
- benchmark/oracle/report semantics;
- provider-specific query APIs.