Blueprint — empirical verification for AI-built LiveView interactions

AI skills, code review rules, and component playbooks reduce the chance of an ownership mistake. They do not prove what a real browser does when a real LiveView patch lands. This project closes that gap with a generation-to-evidence loop:

AI/Developer writes component + interaction contract
                         |
                         v
              real LiveView + real browser
          activate -> patch -> observe -> diagnose
                         |
                         v
                  CI evidence artifact

Final product

The final product is a LiveView Interaction Conformance Kit, not primarily a UI library. It has six layers:

Calibration kernel — this repository's multi-browser, multi-LiveView-version fixtures, Aha trial, red controls, and delegation ledger establish known behavior.
Portable consumer contracts — versioned declarative files describe a route, activation, an acknowledged patch, native probes, and before/after expectations.
Browser execution engine — a strict phase machine drives the consumer's real app and records patch proof, DOM object identity, and native state independently.
Evidence and diagnostics — results distinguish pass, fail, inconclusive, and infrastructure error; explanations are mechanically derived from evidence.
Developer integration — Mix tasks, JSON artifacts, terminal output, and CI browser/version matrices make the contracts part of normal application delivery.
Compatibility knowledge — only after the result schema stabilizes may app-specific evidence feed a broader, versioned ecosystem ledger.

The bundled research ledger and consumer-app evidence remain separate until their semantics genuinely align.

Consumer contract v1

The first public slice deliberately supports one interaction transaction:

visit route
  -> real click/hover or deterministic scroll-into-view activation
  -> verify before-state
  -> retain the subject DOM object
  -> read patch-ack attribute
  -> real click or restricted key press patch trigger
  -> wait for the subject-local attribute to change
  -> read the new DOM object and native state
  -> compare and report

Supported probes are node_identity, attribute, popover, dialog, and focus. Activation supports strict {type, selector} click, hover, and scroll actions. Patch triggers support clicks or a strict {type: "press", selector, key} action whose key is Enter or Space; it does not move the pointer, enabling hover-preserving contracts whose post-state proves actual survival. Hover uses Playwright's normal hit testing and scroll uses scrollIntoViewIfNeeded with no coordinates or script. Contracts are strict JSON: unknown fields, unknown probes, arbitrary JavaScript, timer sleeps, and non-local targets are rejected. A patch that is not acknowledged is inconclusive, not a component failure and never a pass.

Example:

{
  "schema_version": 1,
  "id": "account-menu-survives-refresh",
  "route": "/settings",
  "subject": {"selector": "#account-menu"},
  "activate": [
    {"type": "click", "selector": "#account-menu-trigger"}
  ],
  "patch": {
    "trigger": {"type": "click", "selector": "#refresh-account"},
    "ack": {
      "type": "attribute_change",
      "selector": "#account-menu",
      "attribute": "data-contract-rev"
    }
  },
  "expectations": [
    {"probe": "node_identity", "selector": "#account-menu svg", "after": "same"},
    {
      "probe": "attribute",
      "selector": "#account-menu-trigger",
      "attribute": "aria-current",
      "before": {"value": "true"},
      "after": {"value": "true"}
    },
    {"probe": "popover", "before": {"open": true}, "after": {"open": true}},
    {
      "probe": "focus",
      "selector": "#account-menu-trigger",
      "before": {"active": true},
      "after": {"active": true}
    }
  ]
}

Run it against an already-started local app:

mix live_interaction_contracts.check \
  --url http://127.0.0.1:4000 \
  --contract test/interaction_contracts/account_menu.json

Evidence rules

Patch proof: the acknowledgment element must be the subject or its descendant, and its named attribute must change after the trigger.
Identity: same selector/ID is not enough; the pre-patch and post-patch DOM object references are compared directly. An optional unique selector targets a declared element; without it, identity targets the subject.
Probe targets: node identity, popover, and dialog probes accept optional unique selectors; omitted selectors target the subject. Focus and attribute probes require unique selectors. Probe targets need not be descendants; only patch acknowledgment is subject-local.
Attributes: value is an exact string or null; null asserts absence.
Convergence: activation action targets and declarative before/after expectations are polled within the fixed runner timeout. An activation target or precondition that never arrives is inconclusive; patch-trigger absence retains the existing runner-error behavior. After an acknowledged patch, missing/non-unique probe targets and unmet expectations fail.
Coherence: native probes expose complete relevant vectors. Dialog means open, reflected open attribute, and :modal, not a generic open boolean.
Verdicts: expectation mismatch after an acknowledged patch is fail; failed activation/preconditions or missing acknowledgment is inconclusive; malformed contracts, navigation, and browser failures are error.
Artifacts: every result records contract, runner, Playwright, browser, target, patch acknowledgment, raw observations, diagnostics, and final verdict.

Roadmap

Milestone 1 — external-app vertical slice

mix live_interaction_contracts.check
strict JSON contract v1
Chromium default, one fresh context, one run
click/hover/scroll activation, click or restricted keyboard patch triggers, and subject-local attribute-change acknowledgment
node identity (subject or explicit selector), attribute, popover, dialog, and focus probes
atomic JSON result plus human terminal report
loopback-only by default

Milestone 2 — deterministic consumer suites

first-level directory discovery for independent v1 contracts
complete static preflight before browser launch
deterministic sequential execution with one schema-v1 result per contract
explicit aggregate exit reduction and stale-evidence safety
application-author ownership and contract-writing guide

Milestone 3 — further practical interaction coverage

input value/selection and aria-activedescendant probes
fill, general activation press, and composition-safe actions
Firefox/WebKit CI expansion and optional deterministic repeat
reusable Phoenix test helpers for rendering acknowledgment revisions

Milestone 4 — interaction continuity

insert patches between pointer/key/composition event phases
detect duplicate activation, stale listeners, hook remount, and focus handoff
reconnect and async-result ordering contracts
Playwright traces only as opt-in failure evidence

Milestone 5 — AI verification loop

a skill generates component code and a contract
the agent runs the contract, reads structured diagnostics, and proposes a fix
a fix is accepted only when the same empirical contract passes
contract changes are reviewed like API changes; an agent may not weaken an expectation merely to manufacture green

Milestone 6 — ecosystem evidence

consumer-provided LiveView/app version metadata
version-upgrade comparisons for consumer contracts
opt-in aggregation across component patterns and libraries
real assistive-technology and CJK IME evidence remain separately labelled

Explicit non-goals for v1

no arbitrary JavaScript in contracts
no sleep-based patch completion
no automatic application startup or port killing
no credentials, recorded browser profiles, or production targets by default
no AI auto-fix before raw evidence is available
no claim that same DOM node implies a coherent interaction machine
no merging consumer results into the research ledger yet

← Previous Page Project map