Changelog

View Source

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

0.4.0 - 2026-09-13

Added

  • PureHTML.parse_with_errors/2: parses like parse/2 and returns {nodes, error_count}, where the count follows the WHATWG parse errors emitted by the tokenizer and tree builder
  • scripting: option for PureHTML.parse/2 (default: true)
    • When false, <noscript> content is parsed as HTML instead of raw text
    • Per WHATWG spec, affects in_head, in_body, and in_template insertion modes: <noscript> content is raw text only when scripting is on
  • html5lib tree construction tests now run in both scripting modes per the test README
    • Tests without #script-off/#script-on run in both modes
    • #script-off tests are no longer skipped
    • Filter with --only scripting:on or --only scripting:off

Fixed

  • Fragment parsing: form element pointer now set when context element is <form> (WHATWG spec step 13)
  • Parse error counts now follow the WHATWG rules
    • Tokenizer: end tags with attributes, trailing solidus on non-void start tags, C1 control and unknown named character references, EOF inside script comment-like text
    • Foreign content: HTML and <body> start tags inside foreign content, mismatched foreign end tags; no error at HTML integration points
    • Tables: ignored <tr> and table-structure start tags, mismatched cell end tags, foster-parented characters and end tags (including </p>, which now foster-parents through the in-body rules), an implied cell when the current node is not a cell, <frameset> and <frame> inside a table, and end tags in a nested table, which no longer bypass the in-table rules
    • Column groups and templates: <col> outside a colgroup, <colgroup> in body or inside a template, SVG and HTML breakout inside a colgroup, <tr> after non-table template content, foster-parented tags in a template row, <html> and <a> inside template table content, mismatched </template>, and characters and EOF in frameset, column group, and template contexts
    • Elsewhere: <rp>/<rt> outside ruby, <frame> outside a frameset, </html> and </frameset> on a fragment root, </frameset> in body, caption-closing tokens and mismatched caption end tags, and </form> whose form element is out of scope (the form now stays open, as specified)
  • EOF is dispatched through the insertion modes per the spec instead of being handled once at the end
  • Closing a caption now generates implied end tags first and clears the active formatting list to its marker, so formatting elements opened before the table are reconstructed after it
  • xmp, iframe, and noembed in body now switch to the text insertion mode, so EOF inside them is counted once for the raw text element and once for other open elements
  • Duplicate <a> start tags follow the spec: the adoption agency runs, then the old element is removed from the active formatting list and the stack even when a table put it out of scope; <nobr> uses its own scope-based rule
  • Reconstructed formatting elements are foster-parented when foster parenting applies
  • <frameset> in caption, cell, and table contexts is a parse error and is ignored; it is no longer silently accepted, nor inserted in fragments with no body
  • Query: redundant clauses for namespaced elements removed
  • select is a scope boundary, as in the current standard, so an end tag for an element opened outside a select is ignored inside it
  • <select> inside a table is foster-parented like any other in-body element; <input> in a select-context fragment is ignored; a nested <select> closes the open select; option, optgroup, and hr inside a select generate implied end tags per spec
  • td and th start tags in body are a parse error and ignored; <col> in a table switches to "in column group"; <svg> and <math> in a table go through the in-body rules with foster parenting; the adoption agency uses the shared scope walk
  • Foreign content follows the tree construction dispatcher and the foreign content rules as written: a token at an integration point is processed with the current insertion mode; an HTML breakout start tag, and </br>/</p>, pop out of foreign content and are reprocessed with the current mode; the end-tag walk returns at the topmost node in a fragment; a foreign end tag that reaches in body is any other end tag; the scope walk covers the stack of open elements only
  • A start tag whose self-closing flag is never acknowledged is a parse error for every non-void HTML element, not only those on the generic path; </td> and </th> in body are any other end tag
  • Serializer: < and > are escaped in attribute values, per the spec's "escaping a string" algorithm; the :escape_lt_in_attrs option is removed since the escaping is no longer optional
  • Tokenizer: > right after <!DOCTYPE is only the missing-doctype-name error; an ampersand followed by a name that matches no character reference is an error only when a semicolon ends the name (the ambiguous ampersand state)
  • Table text is collected when the current node is a template; a cell end tag generates implied end tags before checking the current node; character and comment tokens in a row are processed with the in-table rules for that token, so the row is the mode that resumes
  • Head elements (base, link, meta, title, style, script, noframes, template, and noscript with scripting on) are processed with the in-head rules from every mode that delegates to them, and the text insertion mode returns to the mode that opened the element; in body no longer inserts them itself or treats characters under them as raw text by checking the current tag
  • Opening a template pushes "in template" onto the stack of template insertion modes; its end tag pops that entry and resets the insertion mode appropriately, which yields "after head" for the html node once the head element pointer is set
  • li, dd, and dt start tags close an open item by walking the stack as specified: an item of the same kind is closed with implied end tags except itself, and a special element other than address, div, or p ends the walk; menuitem is no longer in the special category
  • rb, rtc, rp, and rt have their own in-body entries: implied end tags (except rtc for rp/rt) with a ruby element in scope, then a parse error unless the current node is what the text names; they no longer close a p
  • A formatting end tag with no entry in the active formatting list is handled by the any-other-end-tag step without an error of its own, on any iteration of the adoption agency's outer loop
  • textarea follows the generic RCDATA algorithm (text insertion mode, a leading line feed dropped there); plaintext switches the tokenizer without leaving the current mode, so a <plaintext> ignored in frameset leaves the tokenizer alone
  • "Close a p element" runs the text's three steps (implied end tags except p, then the current-node check, then the pop), so a p closed over an open option is not an error
  • Frameset-ok is set to "not ok" only by the start tags the text lists; form, noembed, noframes, plaintext, rb, and rtc no longer block a later <frameset>
  • In head's "anything else" pops the current node unconditionally, and EOF in head noscript takes that path too, so a <head> fragment or an open <noscript> at EOF no longer ends up holding the body
  • The active formatting elements are reconstructed from the last marker or still-open entry forward, as specified; popping an element no longer strips its entry from the list
  • Non-whitespace table text is reprocessed through in table's "anything else" entry (parse error, foster parenting, in body rules) instead of a private reconstruction
  • The body start tag follows the text: parse error, ignored with a template on the stack or when the second element of the stack is not a body, otherwise frameset-ok "not ok" and merged attributes
  • Closing a table (or a nested table start tag) pops through the table and resets the insertion mode appropriately, where a template node yields the current template insertion mode
  • The adoption agency follows the text step by step: the early exit for a current node not in the list, the formatting element search bounded by the last marker, lastNode inserted at the appropriate place for the common ancestor (foster parented when foster parenting is on), and the any-other-end-tag step for a missing entry on any iteration
  • A CDATA section is recognized whenever the adjusted current node is not in the HTML namespace, including inside an SVG or MathML integration point such as <svg><title>; it was a bogus comment there
  • The in-body <svg> and <math> entries reconstruct the active formatting elements before inserting the element, as their text says

Removed

  • The "in select" and "in select in table" insertion modes, and "select scope". The standard removed them; select content is parsed with the in-body rules (select, option, optgroup, hr, and input entries, and </select> as a generic block end tag)

Changed

  • Tree builder internals: insertion modes are unary state pipelines; foster parenting is enabled for one token and settled by the tree builder; the in-table "anything else" rules delegate to the in-body rules instead of a second implementation; scope walks take the scope type and state; implied end tags and the in-body delegation helpers live in Helpers
  • The tree builder switches the tokenizer state (RCDATA, RAWTEXT, script data, PLAINTEXT) as the text describes; the tokenizer no longer switches on its own when it emits a start tag, and no longer takes a scripting option
  • Every "process the token using the rules for X" is a plain delegation: the template end tag lives in in head, EOF and <html> in the table-family, column group, frameset, and after head modes go to in body or in head without switching modes, and the in-table delegation no longer saves and restores the caller's mode
  • The insertion parent is the top of the stack of open elements; the separately stored parent pointer is gone
  • The stack of template insertion modes holds only template insertion modes; the table return-mode push/pop is gone
  • In body no longer creates html, head, or body elements on the fly for other modes, and finalize no longer patches a missing head or body in; the insertion modes' EOF rules produce them
  • In body's start tag entries take the token, and the adoption agency works on element refs with one function per step of the text; elements no longer carry a foster-parent marker
  • The selectedcontent mirroring of the customizable select is replayed by PureHTML.TreeBuilder.SelectedContent after parsing, documented as the forms chapter's "update a select's selectedcontent" rather than a tree construction step
  • Foreign content has its own module, PureHTML.TreeBuilder.ForeignContent, mirroring the text's section: the dispatcher decision, the rules for parsing tokens in foreign content, the tokenizer's CDATA test, element insertion with the SVG tag and foreign attribute adjustments, the integration point tests, and breaking out
  • html5lib-tests submodule pinned at 9329e64 (2026-06-22), the last upstream commit with the tree-construction fixtures before they moved to web-platform-tests; it adds the void-in-phrasing fixtures, an adoption case, and corrects <input><option> in a select-context fragment
  • html5lib tree-construction tests count #errors lines only; #new-errors are renamed tokenizer codes, not extra errors
  • html5lib tokenizer and tree-construction suites run one test per fixture file, looping over the cases at run time; the full suite drops from about 68 seconds to under one, since compiling the generated per-case test functions was the cost. HTML5LIB_CASE=file:index runs a single case
  • Tool versions: Erlang 28.4.2 and Elixir 1.20.4; dev dependencies grouped and upgraded

0.3.0 - 2026-02-15

Added

  • Fragment parsing via PureHTML.parse/2 with context: option
    • Implements the WHATWG "parsing HTML fragments" algorithm (used by innerHTML)
    • Context format matches html5lib tests: "div", "body", "svg path", "math mtext"
    • Returns children directly (no <html>/<head>/<body> wrappers)
  • Tree construction dispatcher per WHATWG spec
    • Centralized foreign content routing replaces per-mode ad-hoc checks
    • Adjusted current node handling for fragment parsing contexts
    • Integration point exceptions for MathML text and HTML integration points

Fixed

  • Compile warnings in Query module: grouped apply_combinators/3 clauses and removed duplicate adjacent_sibling_matches/3
  • in_table_body delegation now calls InTable.process directly instead of switching modes, preserving tree construction dispatcher context for foreign content
  • Foster-parented elements no longer incorrectly removed from active formatting list during pop_until_tag
  • Table structure elements (tbody/thead/tfoot/caption/colgroup) handled correctly at foreign integration points in all table-related modes
  • Fragment tokenizer no longer sets last_start_tag, so end tags in escaped script/rawtext/rcdata contexts are correctly treated as character tokens

0.2.0 - 2026-01-20

Added

  • Text extraction with text/2 function
    • Options: :deep, :separator, :strip, :include_script, :include_style, :include_inputs
  • Attribute extraction functions
    • attr/2 - get attribute from single node
    • attribute/2 - extract attribute from list of nodes
    • attribute/3 - query and extract attribute in one step
  • query_one/2 for finding first matching element
  • CSS combinator support in selectors
    • Descendant combinator (space): div p
    • Child combinator: div > p
    • Adjacent sibling combinator: h1 + p
    • General sibling combinator: h1 ~ p

0.1.0 - 2026-01-19

Added

  • CSS selector querying with query/2 and children/2 functions
    • Tag selectors: div, p, a
    • Universal selector: *
    • Class selectors: .class, .foo.bar
    • ID selectors: #id
    • Attribute selectors: [attr], [attr=val], [attr^=prefix], [attr$=suffix], [attr*=substring]
    • Compound selectors: div.class#id[attr]
    • Selector lists: .a, .b
  • get_attr/3 helper for retrieving attribute values from lists
  • merge_attr_lists/2 helper for merging attribute lists (preserves existing values)
  • Serializer options for customizable HTML output (:print_attributes, :escape_comment, :escape_empty)
  • xml_violation_mode for XML infoset coercion in tokenizer
  • All tokenizer initial states enabled in test harness
  • All 23 HTML5 insertion modes implemented:
    • initial, before_html, before_head, in_head, in_head_noscript
    • after_head, in_body, text, in_table, in_table_text
    • in_caption, in_column_group, in_table_body, in_row, in_cell
    • in_select, in_select_in_table, in_template, after_body
    • in_frameset, after_frameset, after_after_body, after_after_frameset
  • HTML5lib tree construction test suite integration (1476 tests)
  • Adoption agency algorithm for proper formatting element handling
  • Active formatting elements list reconstruction
  • Foster parenting for misplaced table content
  • Foreign content support (SVG and MathML namespaces)
  • SVG attribute case adjustments (e.g., viewbox -> viewBox)
  • Template element handling with template mode stack
  • Implicit tag closing per HTML5 specification
  • Leading newline stripping for pre, textarea, and listing elements
  • Ruby element implicit closing (rb, rt, rtc, rp)
  • Second <body> tag attribute merging
  • Nested <form> handling (ignored when form pointer is set)
  • Post-</html> content handling (comments as document siblings)
  • HTML serializer with html5lib-compliant tree output format
  • DOCTYPE quirks mode detection per HTML5 specification

Changed

  • BREAKING: Attributes now use list of tuples instead of maps for Floki compatibility
    • Before: {"p", %{"class" => "foo"}, ["Hi"]}
    • After: {"p", [{"class", "foo"}], ["Hi"]}
    • Attributes are sorted alphabetically for deterministic output
  • Simplified in_table.ex with guard clauses and reduced helpers
  • Consolidated foreign_namespace helper in helpers.ex
  • Refactored tree builder to ref-only stack architecture
    • Stack holds only refs: [ref, ref, ref]
    • Elements map holds all data: ref => %{tag, attrs, children, parent_ref}
  • Consolidated foster parenting into unified foster_parent/2 API with tagged tuples
  • Extracted all insertion modes to separate modules under lib/pure_html/tree_builder/modes/
  • Multi-byte scanning optimization in tokenizer for faster parsing
    • chars_until_null, chars_until_comment, chars_until_cdata
    • Entity reference fast path detection

Fixed

  • Active formatting reconstruction before foster-parenting formatting elements in table context
  • Frameset/noframes mode transitions
  • Ruby nesting edge cases
  • Table handling in foreignObject
  • Multiple tree construction edge cases
  • Original mode preservation for style/script in table context
  • </li> now only closes when li is in list item scope (ul/ol are barriers)
  • <head> in body mode is now properly ignored
  • </br> in table context is foster-parented as <br>
  • MathML mglyph/malignmark namespace preservation in text integration points
  • Foreign content breakout for HTML elements inside SVG/MathML
  • Heading nesting (h1-h6 now properly close previous headings)
  • Frameset handling (body removal from DOM when replaced by frameset)
  • pop_until_one_of boundary element handling for template context
  • </svg> and </math> now properly close all foreign children
  • Foreign content scope boundaries for in_scope? checks (SVG: desc, foreignObject, title; MathML: annotation-xml, mi, mn, mo, ms, mtext)
  • Select mode end tag handling to close pushed HTML elements
  • Void elements (img, br, etc.) treated as self-closing in select mode
  • <selectedcontent> population with first/selected option content
  • Adoption agency for formatting elements inside select mode
  • Active formatting reconstruction before text insertion in select mode
  • </p> handling in table context with foster parenting
  • Option closing when new option/optgroup starts
  • <li> in table context handling
  • HTML5 whitespace handling in various contexts
  • Table/anchor active formatting handling and scope checks
  • Form element pointer handling and nested form detection
  • Row mode bogus <tr> detection
  • current_parent_ref handling in foster parenting contexts
    • Fixed close_tag_ref, close_block_end_tag, close_foreign_root to use stack top
    • Fixed adoption agency to use stack top after popping elements
  • Active formatting reconstruction for void elements in table context
  • Row mode foster parenting with in_body rules delegation