Raxol.UI.SyntaxHighlighter (Raxol v2.6.1)

Structured-token syntax highlighting for terminal rendering.

Components in this codebase emit styled spans (text(content, fg:, style:)) -- raw ANSI is only ever applied at the final Terminal.Renderer stage. A highlighter here must therefore yield structured tokens (%{text:, fg:, styles:}), never a pre-rendered ANSI/HTML string. That ruled out a Shiki port (Oniguruma/TextMate semantics don't translate to Erlang :re), autumn (tree-sitter engine, but only string output formats), and subprocess-based highlighters (bat, chroma).

Built on makeup (pure-Elixir, Pygments-modeled, already a dependency: ExDoc's engine, José Valim/Tiago Barroso). Makeup.Registry covers the native lexers (elixir, erlang, heex, json, html, css, sql, js/ts, gleam, swift, ...) by language name or file extension. If the optional makeup_syntect dependency (Rust syntect NIF, precompiled, MIT) is present, it auto-registers 205 additional syntect syntaxes (python, go, rust, yaml, bash, markdown, ruby, ...) into the same registry on application start -- no extra code here, Makeup.Registry lookups just start finding them.

Unknown languages, missing lexers, or a lexer crash all degrade to a single unstyled token per line (fg: nil) rather than raising -- a highlighting failure must never break a render frame.

Tokenization strategy

Lexers are stateful across lines (a line inside a heredoc or multi-line comment tokenizes as garbage in isolation), so callers must lex the whole file text for a version once, not line-by-line. This module does exactly that: lex/2 runs over the full source, and the resulting flat token stream is split into per-line token lists (a token whose text spans a newline is split into per-line fragments that keep the same token type).

Summary

Types

token()

Functions

highlight_lines(source, language, theme \\ :one_dark)

Tokenizes source as language, returning one token list per line.

Types

token()

@type token() :: %{
  text: String.t(),
  fg: String.t() | nil,
  styles: [:bold | :italic | :underline]
}

Functions

highlight_lines(source, language, theme \\ :one_dark)

@spec highlight_lines(
  String.t(),
  String.t() | nil,
  atom() | Makeup.Styles.HTML.Style.t() | nil
) :: [
  [token()]
]

Tokenizes source as language, returning one token list per line.

language is matched against Makeup.Registry by name first, then by file extension (so both "elixir" and "ex" resolve). nil, an unresolvable language, or a lexer crash all fall back to plain (unstyled) lines -- this function never raises.

theme is either an atom naming one of Makeup's 34 built-in Pygments styles (Makeup.Styles.HTML.StyleMap, e.g. :one_dark, :dracula, :monokai) or an already-resolved %Makeup.Styles.HTML.Style{}. Defaults to :one_dark.

The outer list always has exactly length(String.split(source, "\n")) entries, matching Raxol.UI.Components.Harness.LineDiff's line splitting, so Enum.at(highlight_lines(text, lang, theme), n) lines up with 0-based diff line indices for text.