History Schema (v1)

Copy Markdown View Source

Temper records test outcomes as JSON Lines: one self-contained JSON object per line, UTF-8 encoded, LF-terminated, appended to .temper/history-{partition}.jsonl (see the README for path configuration). This page documents line schema version 1 as a public contract, so that anything reading or writing these files (your own scripts, CI tooling, an aggregation service) can rely on it without reverse-engineering Temper's codec.

The contract is versioned rather than frozen: within schema 1 the format only ever changes additively, and any breaking change arrives as a new schema number, which version-1 consumers already know to skip. The "Versioning" section at the end spells out the rules.

The envelope

Every line carries two envelope fields, and consumers dispatch on both:

FieldTypeMeaning
schemaintegerline format version; this document describes 1
kindstringwhat the line records: "test" or "suite"

Test lines (kind: "test")

One line per finished test. The run context is denormalized into every line, so a single line answers where, when, and on what code a test produced its outcome, with no joins:

{"schema":1,"kind":"test","run_id":"a490cc5edf1e9800b2ca76f59439dba6",
 "at":"2026-08-22T22:54:58Z","sha":"fe892caba20e2ee0a22e2b13d64c17504f91db61",
 "dirty":true,"branch":"feature/formatter-writer","ci":null,"seed":944754,
 "partition":null,"elixir":"1.20.2","otp":"27",
 "module":"Temper.RunContextTest",
 "name":"test new/1 with a full map keeps every gathered value",
 "file":"/home/dev/app/test/temper/run_context_test.exs","line":16,
 "async":true,"test_type":"test","status":"passed","time_us":4,
 "failure":null}

(Line broken here for readability; on disk it is a single line.)

Required fields are always present and never null. Optional fields are written as explicit null when unknown, but consumers should treat an absent key and a null value the same way.

Run context (identical on every line of one run)

FieldTypeRequiredMeaning
run_idstringyes32 lowercase hex characters, random per suite run; groups every line written by one run
atstringyeswhen the run started: ISO 8601 UTC, second precision, Z suffix (2026-08-22T22:54:58Z)
shastring | nullnocommit under test; null when git cannot answer and no TEMPER_SHA was provided
dirtybooleannowhether the working tree had uncommitted changes; a missing value reads as false
branchstring | nullnogit branch, when known
ciobject | nullno{"provider": string, "run_id": string | null}; null outside CI
seedinteger | nullnothe ExUnit seed for this run (non-negative)
partitionstring | nullnoMIX_TEST_PARTITION value; null when unset
elixirstringyesElixir version the suite ran on
otpstringyesOTP major version the suite ran on

Test outcome

FieldTypeRequiredMeaning
modulestringyestest module, as printed by Elixir (MyApp.UserTest)
namestringyesfull test name, including the test prefix ExUnit generates
statusstringyesone of passed, failed, skipped, excluded, invalid
filestring | nullnosource file of the test
lineinteger | nullnoline of the test (positive)
asyncboolean | nullnowhether the test module ran async
test_typestring | nullnoExUnit's test type (test, doctest, ...)
time_usinteger | nullnomeasured runtime in microseconds; null for tests that never ran (e.g. excluded)
failureobject | nullnofailure signature, present only when status is failed (see below)

The pair (module, name) identifies a test across runs; everything else about a test may change from run to run.

The failure signature

For failed tests, failure is:

FieldTypeMeaning
kindstringthe exception module (ExUnit.AssertionError), or the raw kind for non-error exits
messagestringthe failure message, truncated to 500 characters
hashstringlowercase hex of the first 4 bytes of the SHA-256 of the full, untruncated message

The hash exists so equal failures group together even when truncation hides the differing tail: group by (kind, hash) to separate distinct failure modes of one flaky test.

Suite lines (kind: "suite")

One line at the end of each recorded run:

{"schema":1,"kind":"suite","run_id":"a490cc5edf1e9800b2ca76f59439dba6",
 "at":"2026-08-22T22:54:58Z","tests":66,
 "times_us":{"async":60478,"run":434307,"load":null}}
FieldTypeMeaning
run_idstringsame value as the run's test lines
atstringsame value as the run's test lines
testsintegerhow many test lines the run recorded
times_usobject | nullExUnit's suite timing (run, async, load in microseconds; each member may be null)

A run's suite line is the signal that the run completed; test lines without one may come from an interrupted run.

Rules for consumers

  • Never fail a whole file over one line. Skip what you cannot decode, and count skips if you report.
  • Dispatch on schema and kind; silently skip unknown values of either. New kinds may appear within schema 1.
  • Ignore unknown keys on known kinds: schema 1 grows additively.
  • Parse timestamps instead of comparing them as strings. Temper emits second-truncated UTC Z timestamps, but other producers may legally emit fractional seconds or numeric offsets, which misorder under string comparison.
  • A tool that rewrites files must never drop a line merely because it cannot interpret it; only positively corrupt lines (unreadable JSON) may always be dropped. Compaction (mix temper.merge) preserves every non-corrupt line verbatim. Retention (mix temper.clean --older-than / --keep-shas) reads at and sha generically from every line, unknown schemas and kinds included, and drops exactly the lines it can positively match against the requested window: an old suite summary or future-format line ages out like any other, while a line whose relevant field cannot be read survives.
  • Byte-identical lines describe the same recorded event (they can only arise from copying, such as overlapping CI cache restores) and are safe to deduplicate. mix temper.merge does exactly that.

Rules for producers

  • One JSON object per line, UTF-8, LF-terminated, appended only.
  • Stamp schema and kind on every line.
  • Make every line self-contained: full run context, no references to other lines except run_id grouping.
  • Generate run_id as 32 lowercase hex characters, fresh per run; never reuse one across runs.
  • Emit at as second-truncated UTC with a Z suffix.
  • Write required fields always; write optional fields as explicit null rather than inventing placeholder values.

Versioning

  • schema increments only on breaking changes: a removed or retyped field, or changed field semantics.
  • Adding optional fields or new kind values is not a version bump; the consumer rules above already absorb both.
  • Version-1 consumers skip lines with any other schema value, so mixed-version files degrade gracefully: each consumer reads the lines it understands.