PureHTML (PureHTML v0.5.0)

View Source

A pure Elixir HTML5 parser with CSS selector querying.

PureHTML parses HTML strings into a tree of nodes and provides functions to query, traverse, and serialize them back to HTML.

Parsing

iex> PureHTML.parse("<p>Hello</p>")
[{"html", [], [{"head", [], []}, {"body", [], [{"p", [], ["Hello"]}]}]}]

Querying

iex> html = PureHTML.parse("<div><p class='intro'>Hello</p></div>")
iex> PureHTML.query(html, ".intro")
[{"p", [{"class", "intro"}], ["Hello"]}]

Serializing

iex> [{"p", [], ["Hello"]}] |> PureHTML.to_html()
"<p>Hello</p>"

Node Format

HTML elements are {tag, attrs, children} tuples, the same shape Floki uses. Foreign elements, namespaced attributes, template content, and processing instructions are this parser's.

  • {tag, attrs, children} — HTML element

  • {{:svg | :math, local}, attrs, children} — foreign element

  • {:content, children} — sole child of template
  • {:doctype, name, public_id, system_id} — name is nil when missing
  • {:comment, text}
  • {:pi, target, data}
  • "text"

Attributes are {name, value} tuples in source order. name is a string or {:xlink | :xml | :xmlns, local}. A duplicate name keeps the first value.

Summary

Types

attr()

One attribute: name and value, in source order on the element.

attr_name()

Attribute name: a local name, or a namespaced name from the adjust-foreign-attributes table.

html_node()

A node parse/2 returns. The runtime check of this type is valid_node?/1 in test/pure_html_test.exs.

Functions

attr(node, name)

Extracts an attribute value from a single node.

attribute(nodes, name)

Extracts attribute values from a list of nodes.

attribute(html, selector, name)

Finds elements matching a selector and extracts an attribute from them.

children(node, opts \\ [])

Returns the immediate children of a node.

parse(html, opts \\ [])

Parses an HTML string into a list of nodes.

parse_with_errors(html, opts \\ [])

Parses HTML like parse/2 and returns {nodes, error_count}.

query(html, selector)

Finds all nodes matching the CSS selector.

query!(html, selector)

Like query/2, but raises ArgumentError on an invalid selector.

query_one(html, selector)

Finds the first node matching the CSS selector.

query_one!(html, selector)

Like query_one/2, but raises ArgumentError on an invalid selector.

text(html, opts \\ [])

Extracts text content from an HTML tree or node.

to_html(nodes, opts \\ [])

Converts parsed HTML nodes back to an HTML string.

Types

attr()

@type attr() :: {attr_name(), String.t()}

One attribute: name and value, in source order on the element.

attr_name()

@type attr_name() :: String.t() | {:xlink | :xml | :xmlns, String.t()}

Attribute name: a local name, or a namespaced name from the adjust-foreign-attributes table.

html_node()

@type html_node() ::
  {String.t(), [attr()], [html_node()]}
  | {{:svg | :math, String.t()}, [attr()], [html_node()]}
  | {:content, [html_node()]}
  | {:doctype, String.t() | nil, String.t() | nil, String.t() | nil}
  | {:comment, String.t()}
  | {:pi, String.t(), String.t()}
  | String.t()

A node parse/2 returns. The runtime check of this type is valid_node?/1 in test/pure_html_test.exs.

Functions

attr(node, name)

Extracts an attribute value from a single node.

Returns the attribute value as a string, or nil if the attribute doesn't exist or the input is not an element.

Examples

iex> node = {"a", [{"href", "/home"}, {"class", "link"}], ["Home"]}
iex> PureHTML.attr(node, "href")
"/home"

iex> node = {"a", [{"href", "/home"}], ["Home"]}
iex> PureHTML.attr(node, "title")
nil

iex> PureHTML.attr("text node", "href")
nil

attribute(nodes, name)

Extracts attribute values from a list of nodes.

Returns a list of attribute values. Nodes without the attribute are skipped.

Examples

iex> "<a href='/one'>One</a><a href='/two'>Two</a>"
...> |> PureHTML.parse()
...> |> PureHTML.query("a")
...> |> PureHTML.attribute("href")
["/one", "/two"]

attribute(html, selector, name)

Finds elements matching a selector and extracts an attribute from them.

Combines query/2 and attribute/2 into a single call for convenience. This is the most common pattern for scraping.

Examples

iex> "<nav><a href='/'>Home</a><a href='/about'>About</a></nav>"
...> |> PureHTML.parse()
...> |> PureHTML.attribute("a", "href")
["/", "/about"]

iex> "<div><img src='a.png'><img src='b.png'></div>"
...> |> PureHTML.parse()
...> |> PureHTML.attribute("img", "src")
["a.png", "b.png"]

children(node, opts \\ [])

Returns the immediate children of a node.

Options

  • :include_text - Include text nodes (default: true)

Examples

iex> node = {"div", [], [{"p", [], ["Hello"]}, "Some text"]}
iex> PureHTML.children(node)
[{"p", [], ["Hello"]}, "Some text"]

iex> node = {"div", [], [{"p", [], ["Hello"]}, "Some text"]}
iex> PureHTML.children(node, include_text: false)
[{"p", [], ["Hello"]}]

parse(html, opts \\ [])

@spec parse(
  String.t(),
  keyword()
) :: [html_node()]

Parses an HTML string into a list of nodes.

Returns a list of html_node/0. A doctype, if present, is first.

Options

  • :context - Context element for fragment parsing (e.g., "div", "svg title")
  • :scripting - Whether the scripting flag is enabled (default: true). When false, <noscript> content is parsed as HTML instead of raw text.

Examples

iex> PureHTML.parse("<p>Hello</p>")
[{"html", [], [{"head", [], []}, {"body", [], [{"p", [], ["Hello"]}]}]}]

iex> PureHTML.parse("<!DOCTYPE html><html></html>")
[{:doctype, "html", nil, nil}, {"html", [], [{"head", [], []}, {"body", [], []}]}]

iex> [{"html", [], [{"head", [], []}, {"body", [], [svg]}]}] = PureHTML.parse("<svg></svg>")
iex> svg
{{:svg, "svg"}, [], []}

parse_with_errors(html, opts \\ [])

@spec parse_with_errors(
  String.t(),
  keyword()
) :: {[html_node()], non_neg_integer()}

Parses HTML like parse/2 and returns {nodes, error_count}.

The count follows the parse errors the WHATWG tokenizer and tree construction rules define (a missing doctype, an unexpected end tag, an element still open at the end of the input, and so on). Well-formed input yields 0. Takes the same options as parse/2.

Examples

iex> PureHTML.parse_with_errors("<!DOCTYPE html><p>Hi</p>")
{[{:doctype, "html", nil, nil}, {"html", [], [{"head", [], []}, {"body", [], [{"p", [], ["Hi"]}]}]}], 0}

iex> {_nodes, errors} = PureHTML.parse_with_errors("<p>Hi")
iex> errors
1

query(html, selector)

Finds all nodes matching the CSS selector.

Supported Selectors

  • Tag: div, p, a
  • Universal: *
  • Class: .class
  • ID: #id
  • Attribute: [attr], [attr=value], [attr^=prefix], [attr$=suffix], [attr*=substring]
  • Selector list: .a, .b

An invalid selector represents, and therefore matches, nothing: the result is []. query!/2 raises instead.

Examples

iex> html = PureHTML.parse("<div><p class='intro'>Hello</p><p>World</p></div>")
iex> PureHTML.query(html, "p.intro")
[{"p", [{"class", "intro"}], ["Hello"]}]

iex> html = PureHTML.parse("<ul><li>A</li><li>B</li></ul>")
iex> PureHTML.query(html, "li")
[{"li", [], ["A"]}, {"li", [], ["B"]}]

query!(html, selector)

Like query/2, but raises ArgumentError on an invalid selector.

query_one(html, selector)

Finds the first node matching the CSS selector.

Returns the first matching node in document order, or nil if no match is found. Stops at the first match, so it is cheaper than query/2 when you only need one result. An invalid selector matches nothing.

Examples

iex> "<ul><li>A</li><li>B</li></ul>"
...> |> PureHTML.parse()
...> |> PureHTML.query_one("li")
{"li", [], ["A"]}

iex> "<div><p>Hello</p></div>"
...> |> PureHTML.parse()
...> |> PureHTML.query_one(".missing")
nil

query_one!(html, selector)

Like query_one/2, but raises ArgumentError on an invalid selector.

text(html, opts \\ [])

Extracts text content from an HTML tree or node.

Options

  • :deep - Traverse all descendants (default: true). When false, only direct text children.
  • :separator - String to insert between text segments (default: "")
  • :strip - Strip whitespace from each segment and remove empty segments (default: false)
  • :include_script - Include text from <script> tags (default: false)
  • :include_style - Include text from <style> tags (default: false)
  • :include_inputs - Include value from <input> and <textarea> (default: false)

Examples

iex> html = PureHTML.parse("<p>Hello <strong>World</strong></p>")
iex> PureHTML.text(html)
"Hello World"

iex> html = PureHTML.parse("<ul><li>A</li><li>B</li></ul>")
iex> PureHTML.text(html, separator: ", ")
"A, B"

iex> html = PureHTML.parse("<div>Hello<script>alert(1)</script>World</div>")
iex> PureHTML.text(html)
"HelloWorld"

iex> html = PureHTML.parse("<form><input value='test'></form>")
iex> PureHTML.text(html, include_inputs: true)
"test"

Formatted HTML often contains whitespace for indentation. Use :strip to clean it:

iex> html = PureHTML.parse("<ul>\n  <li>One</li>\n  <li>Two</li>\n</ul>")
iex> PureHTML.text(html, strip: true, separator: " | ")
"One | Two"

to_html(nodes, opts \\ [])

@spec to_html(
  [html_node()],
  keyword()
) :: String.t()

Converts parsed HTML nodes back to an HTML string.

Follows the HTML fragment serialization algorithm.

Options

  • :scripting — whether scripting is enabled (default: true).

    to_html/2 is usually called on a tree parse/2 built. Parse's scripting flag decides whether noscript children are a text node or elements. With the default parse (scripting on), noscript holds one text node that must be emitted literally, so the serializer default is true. If the tree was parsed with scripting off, noscript holds elements and this flag does not affect them.

Examples

iex> PureHTML.parse("<p>Hello</p>") |> PureHTML.to_html()
"<html><head></head><body><p>Hello</p></body></html>"

iex> PureHTML.to_html([{"div", [{"class", "foo"}], ["text"]}])
"<div class=\"foo\">text</div>"