Roux.Code (roux v0.2.2)

The code a computation runs, named by a digest: what a query's code version is made of (Roux.Query's code: option).

A computation runs the code its remote calls reach from where it starts. closure/2 reads that reach from each module's import table, from roots onward, through the project and its dependencies, and stops at the runtime: OTP's modules and Elixir's, whose versions a digest carries instead, and consolidated protocols, which are the build's dispatch tables. digest/2 hashes what it found, each module by beam_digest/2.

So an edit to a module moves the digest of every root set that reaches it, and of no other. A call the import table does not show — apply/3, module.fun() on a module held in a variable — reaches code the walk cannot see: name what it can reach as a root of its own. Roux.Code.Verify.executed/2 runs a computation with call counting on, for a test that checks the closure against what actually ran.

Where code is read

Object code comes from :code.get_object_code/1, which finds a module on the code path wherever it lives: a .beam file, or an escript's archive. A module that was compiled in memory (or cover-compiled) has no object code to read, and no digest can name it: closure/2 returns an error naming it.

Memoized per VM

A closure and its digest are computed once per VM for each set of roots and options: a VM that loads new code mid-run keeps the digest of the code it started with (forget/0 drops them).

Kept across VMs

With store:, digest/2 keeps a digest in a Roux.Blob store as a verifying trace (Roux.Blob.Trace) over what computing it read: where every module the walk met resolves (:code.which/1, one observation of them all), the stat stamp (size, modification time, inode, change time) of every file the walk read object code from, and the absence of every module it found absent. A fresh VM whose beams have not moved gets the digest from the code server and a few stat calls, reading no beam at all. A module in an archive — an escript's — has no stamp of its own, and is stamped by the archive: a fresh run of an escript verifies one stat. A trace is not kept when a file was written within the last two seconds (a stamp that young may not yet show a write that follows it).

The stamps alone would say only that the files the walk read are unchanged, not that a walk now would read them: builds that share a store — two checkouts, an application renamed with its old ebin left in _build, an escript beside a project — each leave their files in place, and a trace over one build's files verified in another served the other build's digest. Where each module resolves tells them apart.

In an escript

An escript's modules live in its archive, and so does Elixir's standard library when the escript embeds it (as mix escript.build does): Elixir's :code.lib_dir/1 is then a path inside the escript. Elixir's modules are recognized by their application wherever they live, and everything else in the archive is code like any other.

Summary

Types

location()

Where a module of a closure was read from, or :absent (not on the code path).

option()

Options of closure/2 and digest/2

Functions

beam_digest(beam, opts \\ [])

A digest of a compiled module that names what it does, not where it was built.

build_root(beam)

The directory holding the innermost _build on beam's path, or nil when there is none.

canonical_beam(beam)

Rebuilds beam without its ExCk and Docs chunks.

closure(roots, opts \\ [])

The modules roots reach, sorted, each with the file its object code was read from, or :absent for a module a call names that is not on the code path (an optional dependency: its arrival would change what the call does). Runtime modules are not in it (see the moduledoc).

digest(roots, opts \\ [])

A digest of closure/2's code: each module by name and beam_digest/2 (or as absent), and the runtime's version (runtime_version/0). Lowercase hex.

forget()

Drops every closure and digest this VM memoized.

runtime_version()

The version of the runtime a digest covers instead of its code: the OTP release, the ERTS version and Elixir's version.

Types

location()

@type location() :: Path.t() | :absent

Where a module of a closure was read from, or :absent (not on the code path).

option()

@type option() ::
  {:exclude, [module()] | (module() -> boolean())}
  | {:follow_excluded, boolean()}
  | {:store, Roux.Blob.t() | nil}

Options of closure/2 and digest/2:

:exclude — modules left out of the closure, as a list or a predicate: a module whose meaning a caller keys some other way (argus's schema, keyed on the entries a query read of it).
:follow_excluded — whether the walk goes on through an excluded module to what it calls (default true): its callees are code like any other. false stops there.
:store — a Roux.Blob store to keep the digest in across VMs (digest/2 only; see "Kept across VMs").

Functions

beam_digest(beam, opts \\ [])

@spec beam_digest(
  Path.t() | binary(),
  keyword()
) :: {:ok, binary()} | {:error, term()}

A digest of a compiled module that names what it does, not where it was built.

A beam's bytes carry the absolute path of the tree it was compiled in: the source file in its compile info and debug info, and — when the code reads __DIR__, __ENV__.file or Application.app_dir/2 at compile time — in its literals too. Two worktrees at one commit build byte-different beams of the same code. This digest reads the chunks instead, with the build root taken out.

Every chunk is hashed, in chunk-id order, except the ones that describe the code rather than being part of it: CInf (compile options and the source path), Docs (prose) and ExCk (the Elixir type checker's export table) never are; Dbgi and Abst (debug info) only with debug_info: true. Line stays: an edit that moves a line moves the digest.

The build root is the directory holding the innermost _build on the beam's path (build_root/1), or root: for a binary. Inside the literal table, the attributes, the line table's file names and the debug info, every binary containing it — and every charlist starting with it — has it replaced by $ROOT. A fun's OldUniq and the vsn attribute (the default one is the module's MD5) are cleared: neither changes what the code computes.

Returns the raw SHA-256, or why :beam_lib could not read the beam.

build_root(beam)

@spec build_root(Path.t()) :: String.t() | nil

The directory holding the innermost _build on beam's path, or nil when there is none.

canonical_beam(beam)

@spec canonical_beam(binary()) :: binary()

Rebuilds beam without its ExCk and Docs chunks.

Elixir rewrites a module's beam when a compile-time dependency is recompiled even if nothing in the module changed: ExCk (the type checker's signature cache) is regenerated, and Docs moves with any @doc edit. For a consumer that reads neither, two beams that differ only there are the same input: hash this instead of the bytes. A binary :beam_lib cannot parse comes back unchanged.

closure(roots, opts \\ [])

@spec closure([module()], [option()]) ::
  {:ok, [{module(), location()}]} | {:error, {:no_beam, module()}}

digest(roots, opts \\ [])

@spec digest([module()], [option()]) :: {:ok, String.t()} | {:error, term()}

A digest of closure/2's code: each module by name and beam_digest/2 (or as absent), and the runtime's version (runtime_version/0). Lowercase hex.

forget()

@spec forget() :: :ok

Drops every closure and digest this VM memoized.

runtime_version()

@spec runtime_version() :: String.t()

The version of the runtime a digest covers instead of its code: the OTP release, the ERTS version and Elixir's version.