Lockstep.MixCompiler.Preprocessors (Lockstep v0.1.0)

Copy Markdown View Source

Built-in source-string preprocessors for Lockstep.MixCompiler.compile/1.

Many real-world Elixir libraries do compile-time work that breaks when the source file is relocated (which is what Lockstep.MixCompiler does — it rewrites into _build/.../lockstep_rewritten/). The most common offenders:

  • @moduledoc "README.md" |> File.read!() |> ... — fails because the relocated file's cwd doesn't contain README.md (nimble_pool, highlander).
  • Code.ensure_loaded!(SomeMod) at module level — fails when SomeMod is a sibling that hasn't been loaded yet (Hammer's macro-based dispatch).
  • @external_resource "..." — typically harmless but can surface as a parse error if the path is computed at compile time.

Each preprocessor here is a (source_string, path) -> source_string function compatible with the :preprocess option of Lockstep.MixCompiler.compile/1.

Summary

Functions

Apply the safe set of preprocessors: strip_compile_time_external_reads/2 and strip_code_ensure_loaded/2. Useful as a default for big libs.

Inline a module-level cond whose arms gate on Code.ensure_loaded?(<json_lib>). Targets the common Elixir pattern of falling back between OTP 27's JSON and Jason

Replace @<attr> for ... Code.ensure_loaded?(mod) ... do: mod (a comprehension that builds an attribute list of available optional error modules) with a fixed empty list.

Expand local aliases that collide with stdlib module names targeted by Lockstep.Rewriter.

Comment out Code.ensure_loaded!(...) calls that appear at module level (typical in __before_compile__ macros that gate on a sibling module being loaded).

Replace @<attr> <expression-that-reads-an-external-file> with a static string. Targets two common patterns

Unwrap module-level guards of the form

Functions

defaults(source, path)

@spec defaults(String.t(), String.t()) :: String.t()

Apply the safe set of preprocessors: strip_compile_time_external_reads/2 and strip_code_ensure_loaded/2. Useful as a default for big libs.

inline_json_dispatch(source, path)

@spec inline_json_dispatch(String.t(), String.t()) :: String.t()

Inline a module-level cond whose arms gate on Code.ensure_loaded?(<json_lib>). Targets the common Elixir pattern of falling back between OTP 27's JSON and Jason:

cond do
  Code.ensure_loaded?(JSON) -> defdelegate ... to: JSON
  Code.ensure_loaded?(Jason) -> defdelegate ... to: Jason
  true -> IO.warn(...)
end

Replaces the whole cond do ... end block with delegates to a fixed module (default: Jason). Useful only for files we know follow this exact shape; otherwise leaves the source unchanged.

inline_optional_module_list(source, path)

@spec inline_optional_module_list(String.t(), String.t()) :: String.t()

Replace @<attr> for ... Code.ensure_loaded?(mod) ... do: mod (a comprehension that builds an attribute list of available optional error modules) with a fixed empty list.

This is conservative: if you actually want one of those error modules in the rewritten environment, supply your own list via the replacement keyword (default: empty list literal []).

preserve_aliased_module_names(source, path)

@spec preserve_aliased_module_names(String.t(), String.t()) :: String.t()

Expand local aliases that collide with stdlib module names targeted by Lockstep.Rewriter.

The rewriter doesn't track alias declarations: it sees a bare Registry.select(...) and rewrites to Lockstep.Registry.select, even when an alias MyApp.{... Registry ...} at the top of the file meant the call to resolve to MyApp.Registry. The fix is to textually rewrite Registry.<f>(...) to <NS>.Registry.<f>(...) in source files that alias <NS>.Registry.

Currently handles:

  • alias <NS>.Registry (single import) and alias <NS>.{...Registry...} (multi-import) — rewrites bare Registry.<f>(...) calls to <NS>.Registry.<f>(...) so the rewriter's module_matches?(callee, Registry) returns false.

Other colliding names in Lockstep.Rewriter's target set (GenServer, Supervisor, Task, Agent, Process) aren't yet handled by this preprocessor.

strip_code_ensure_loaded(source, path)

@spec strip_code_ensure_loaded(String.t(), String.t()) :: String.t()

Comment out Code.ensure_loaded!(...) calls that appear at module level (typical in __before_compile__ macros that gate on a sibling module being loaded).

When Lockstep recompiles a rewritten file out of order, the gated module may not yet be available, causing a compile error. Stripping the assertion lets compilation proceed; the user is responsible for loading dependencies in the right order via their setup_all.

Conservative: only matches lines whose stripped form starts with Code.ensure_loaded!. Does not match calls inside function bodies (which are runtime, not compile-time).

strip_compile_time_external_reads(source, path)

@spec strip_compile_time_external_reads(String.t(), String.t()) :: String.t()

Replace @<attr> <expression-that-reads-an-external-file> with a static string. Targets two common patterns:

# Pattern 1 (nimble_pool, highlander):
@moduledoc "README.md"
           |> File.read!()
           |> String.split("<!-- MDOC !-->")
           |> Enum.fetch!(1)

# Pattern 2 (footer-style):
@doc_footer readme
            |> File.read!()
            |> String.split("<!-- MDOC -->")
            |> Enum.fetch!(1)

After preprocessing, the attribute becomes a placeholder noting that the file was rewritten by Lockstep. Behavior is unaffected because the attribute's only consumer is @moduledoc/@doc, neither of which is load-bearing at runtime.

Match heuristic

Looks for @<attr> followed by a multi-line expression containing File.read!. Replaces the entire @<attr> ... through the line ending the pipeline. Conservative — when in doubt, leaves the source unchanged. Handles multiple matches in one file.

unwrap_optional_dep_guards(source, path)

@spec unwrap_optional_dep_guards(String.t(), String.t()) :: String.t()

Unwrap module-level guards of the form

if Code.ensure_loaded?(SomeMod) do
  defmodule MyModule do
    ...
  end
end

Many libraries gate an entire module on whether an optional dependency is loaded (e.g. if Code.ensure_loaded?(Postgrex) do ... end around a Postgrex-only module). When Lockstep recompiles the source against a different dep set, this guard evaluates to false and the module is silently not defined, causing puzzling module X is not available errors later.

Removes the guard wrapper: the inner defmodule is always defined. Conservative: only matches the exact if Code.ensure_loaded?(...) do

  • matching final end pattern at file top level.