ExAthena.Provenance (ExAthena v0.19.0)

Derive what a run actually did from its own tool calls.

A worker's report is free text, so an orchestrator reading it cannot distinguish "the worker ran mix compile" from "the worker said it compiled". Live runs produced exactly that failure: a deliverable claiming "the app compiles cleanly with no new errors" for a run in which no build command was ever executed.

These events are read off the message history rather than the prose, so the claim becomes checkable — both by the model (the footer rides back with the worker's report) and by deterministic rails (changed_files/1, commands/1).

Three event kinds are recorded, in call order:

{:write, path} — a file the run mutated.
{:command, cmd} — a shell command that ran and exited zero.
{:failed_command, cmd} — one that ran and exited non-zero. Bash returns {:ok, …} whatever the exit code, so without this a worker could watch the suite go red and still satisfy a rail asking whether the change was exercised.

Order is preserved because it carries information a set cannot: "wrote a test, ran it, then edited source" and "edited source, then added a test" have the same files and commands but very different meanings — see test_first?/1.

Summary

Types

event()

Functions

changed_files(events)

Distinct files the run mutated, in first-seen order.

commands(events)

Distinct commands the run executed, in first-seen order — including ones that exited non-zero, which still ran.

events(messages, opts \\ [])

Ordered events derived from a run's messages.

failed_commands(events)

Distinct commands that exited non-zero.

footer(events)

A one-line factual summary to append to a worker's report, or nil when the worker neither changed nor ran anything (a pure read-only explorer).

scan(text)

Recover events from footers embedded in a transcript.

test_command?(cmd)

Whether a command runs a test suite.

test_file?(path)

Whether a path looks like a test rather than production code.

test_first?(events)

Whether a test file was written before the first source file.

Types

event()

@type event() ::
  {:write, String.t()} | {:command, String.t()} | {:failed_command, String.t()}

Functions

changed_files(events)

@spec changed_files([event()]) :: [String.t()]

Distinct files the run mutated, in first-seen order.

commands(events)

@spec commands([event()]) :: [String.t()]

Distinct commands the run executed, in first-seen order — including ones that exited non-zero, which still ran.

events(messages, opts \\ [])

@spec events(
  [ExAthena.Messages.Message.t()],
  keyword()
) :: [event()]

Ordered events derived from a run's messages.

Options:

:mutating_tools — extra tool names (custom/MCP) to treat as writes. Their path is read from a path/file_path argument when present.

failed_commands(events)

@spec failed_commands([event()]) :: [String.t()]

Distinct commands that exited non-zero.

A rail asking "was this change exercised?" must not be satisfied by a suite that ran and went red.

scan(text)

@spec scan(String.t()) :: [event()]

Recover events from footers embedded in a transcript.

A tool cannot write into loop state, so the footer is the only channel by which an orchestrator's rails learn what its workers did — the text is a parsing contract, not merely display. Prose around the footers is ignored, and an elided (+N more) remainder simply does not come back.

test_command?(cmd)

@spec test_command?(String.t()) :: boolean()

Whether a command runs a test suite.

A build is not a test: the failure this exists for compiled cleanly and raised on every page load, so "it compiles" must not satisfy a rail asking whether anything actually exercised the change.

test_file?(path)

@spec test_file?(String.t()) :: boolean()

Whether a path looks like a test rather than production code.

Either it lives under a test directory, or its basename carries a test affix. Deliberately ecosystem-spanning: the rails that use this run against whatever project the agent was pointed at, not just Elixir.

test_first?(events)

@spec test_first?([event()]) :: boolean()

Whether a test file was written before the first source file.

This is the one question a set of events cannot answer, and the reason events/1 preserves order: "wrote a test, ran it red, then implemented" and "implemented, then bolted a test on" touch the same files and run the same commands. Vacuously true when no source was written.