MobDev.Smoke (mob_dev v0.7.21)

Copy Markdown View Source

The decisions behind mix mob.smoke, kept free of I/O so each can be tested against the shapes agent-device and Mob.Diag really return.

A replayed flow passing is the tool's opinion. The app's own counters say whether it held up: a store that lost its table or reset, events the listener could not deliver, or, on mob 0.9.7 or later, no receipts at all while a flow ran (the taps landed somewhere other than this app). So the verdict combines both. A flow whose health could not be read is reported as not checked: it does not fail the run, but the run does not claim the app held up either.

agent-device test --json reports "success": true even when scripts fail; only data.failed and data.notRun say what happened.

The counters live in the app's BEAM and start from zero when it boots, and a flow that begins with open --relaunch boots a new one. So health is read around each flow, and a snapshot from a different BEAM (its OS pid changed) is compared against zero rather than against the old BEAM's counters.

Summary

Types

claim()

A session's lease on a device.

cli_error()

agent-device refused to run ("success": false).

failure()

One failed script from agent-device test --json.

finding()
outcome()
reply()

An RPC reply sorted by what it means for the check.

report()

The parts of a suite report the verdict reads.

result()

Everything mix mob.smoke learned about one device. health is present when health was checked flow by flow: how many flows ran, and why each flow whose snapshot before or after it did not read went unchecked.

snapshot()

Mob.Diag.health/0 and the BEAM's OS pid, read together.

Types

claim()

@type claim() :: MobDev.DeviceLeases.claim()

A session's lease on a device.

cli_error()

@type cli_error() :: %{
  code: String.t() | nil,
  message: String.t(),
  hint: String.t() | nil
}

agent-device refused to run ("success": false).

failure()

@type failure() :: %{
  file: String.t() | nil,
  status: String.t() | nil,
  attempts: non_neg_integer(),
  artifacts_dir: String.t() | nil,
  code: String.t() | nil,
  cause: String.t() | nil,
  message: String.t() | nil,
  hint: String.t() | nil
}

One failed script from agent-device test --json.

finding()

@type finding() :: {:failure | :warning | :note, String.t()}

outcome()

@type outcome() :: {:blocked, String.t()} | {:ran, report(), integer()}

reply()

@type reply() ::
  {:ok, term()}
  | {:unreachable, String.t()}
  | {:unsupported, String.t()}
  | {:error, String.t()}

An RPC reply sorted by what it means for the check.

report()

@type report() :: %{
  total: non_neg_integer(),
  executed: non_neg_integer(),
  passed: non_neg_integer(),
  failed: non_neg_integer(),
  skipped: non_neg_integer(),
  not_run: non_neg_integer(),
  duration_ms: non_neg_integer(),
  failures: [failure()]
}

The parts of a suite report the verdict reads.

result()

@type result() :: %{
  :device => String.t(),
  :outcome => outcome(),
  :findings => [finding()],
  :receipts_delta => integer() | nil,
  optional(:health) => %{flows: non_neg_integer(), unchecked: [String.t()]}
}

Everything mix mob.smoke learned about one device. health is present when health was checked flow by flow: how many flows ran, and why each flow whose snapshot before or after it did not read went unchecked.

snapshot()

@type snapshot() :: %{health: reply(), beam: reply()}

Mob.Diag.health/0 and the BEAM's OS pid, read together.