HTML parsing on top of gumbo-parser 0.14.1.
parse/2 and parse_fragment/3 return an opaque Gumbo.Output handle
backed by a NIF resource: the tree stays native and is walked lazily
through Gumbo.Node accessors. The handle holds a private copy of the
input and is freed when it is garbage collected.
Summary
Functions
Converts a handle, a node, a list of nodes, or nil into plain nested
Elixir data in a single NIF call.
Parses html and returns a Gumbo.Output handle.
Like parse/2, but raises ArgumentError with the reason on failure.
Parses html as a fragment whose context element is context, returning a
Gumbo.Output handle.
Like parse_fragment/3, but raises ArgumentError with the reason on
failure.
Functions
@spec materialize( Gumbo.Output.t() | Gumbo.Node.t() | [Gumbo.Node.t()] | nil, keyword() ) :: term()
Converts a handle, a node, a list of nodes, or nil into plain nested
Elixir data in a single NIF call.
The walk is iterative in C — the node table is preorder, and each node's map is built from its already-built children in one reverse pass — so depth is bounded by memory, not by the scheduler's C stack.
Gumbo.Output.t() materializes to %{document: ..., root: ..., errors: [...]}, Gumbo.Node.t() to that node's map, a list to the list of its
nodes' maps, and nil to nil. root is nil for a
stop_on_first_error parse that halted before any element was built —
that handle has no root, only errors.
detail: :full (the default) carries every gumbo field, with nil where
the node type has none; detail: :tree returns the tree a caller processes
rather than queries — each element as %{type:, tag_name:, attributes:, children:}, with the name as written and the attributes in source order as
{name, value} pairs whose prefix the parser split off is composed back on —
and every other node as :content has it; detail: :content returns a usable
tree without the source bookkeeping (positions, originals, flags and index) —
except that an element whose tag is :unknown also carries tag_name,
because its atom is the one thing that cannot name it, and a processing
instruction also carries target and data,
the two parts the spec gives that payload. The
materialized :full values agree with the lazy accessors — they come
from the same extractors — under the materialized shape's own key names
(start_position, name).
detail: is validated here and raises ArgumentError for anything but
:full | :tree | :content.
Like parse/2, materializations of 20 480 nodes or more run on a dirty
CPU scheduler — the size proxy is the handle's cached node count, so the
dispatch costs no NIF call — with the same dirty: override; it raises
ArgumentError for anything but :auto | true | false.
Examples
iex> {:ok, output} = Gumbo.parse("<!doctype html><p>x</p>")
iex> Gumbo.materialize(output) |> Map.keys() |> Enum.sort()
[:document, :errors, :root]
iex> {:ok, output} = Gumbo.parse("<!doctype html><p>x</p>")
iex> Gumbo.materialize(output, detail: :content)[:root][:tag]
:html
iex> {:ok, output} = Gumbo.parse_fragment(~s(<p class="a" id="b">x</p>), :body)
iex> [p] = Gumbo.Output.fragment_nodes(output)
iex> Gumbo.materialize(p, detail: :tree)[:attributes]
[{"class", "a"}, {"id", "b"}]
@spec parse(binary(), keyword()) :: {:ok, Gumbo.Output.t()} | {:error, term()}
Parses html and returns a Gumbo.Output handle.
options are validated by Gumbo.Options.new/1; an unknown key or an
out-of-range value returns {:error, {:invalid_option, key, value}}.
Inputs over 16 MiB are rejected with {:error, :input_too_large}: the
parse recurses on the input's nesting depth, so the C stack is sized from
the input's length, and that limit is where the reservation stays
reasonable. An element with more than :max_attributes attribute names (1024
by default, a repeated name included) is rejected with
{:error, :too_many_attributes} rather than having
the extra attributes dropped, because a tree missing attributes would be one
this library's fidelity claim does not cover. A document past :max_tree_depth
open elements (no limit by default) is rejected with {:error, :tree_too_deep}
rather than handed back truncated, for the same reason. A document that runs the parse out
of the C stack it was given is rejected with {:error, :too_deep}: the stack is
sized from the input's length, so this is a backstop under the arithmetic rather
than a limit a caller has to think about.
Parses of 20 KiB or more (Gumbo.Options.dirty_threshold/0 bytes) run
on a dirty CPU scheduler so a large document does not block a normal
scheduler; dirty: false forces the normal scheduler, dirty: true
the dirty one. On a VM without dirty schedulers the dirty choice falls
back to the normal path invisibly.
Examples
iex> {:ok, output} = Gumbo.parse("<!doctype html><p>x</p>")
iex> Gumbo.Node.type(Gumbo.Output.root(output))
:element
@spec parse!(binary(), keyword()) :: Gumbo.Output.t()
Like parse/2, but raises ArgumentError with the reason on failure.
Examples
iex> Gumbo.parse!("<!doctype html><p>x</p>") |> Gumbo.Output.node_count()
6
@spec parse_fragment(binary(), Gumbo.Tag.t(), keyword()) :: {:ok, Gumbo.Output.t()} | {:error, term()}
Parses html as a fragment whose context element is context, returning a
Gumbo.Output handle.
Gumbo has no gumbo_parse_fragment: a fragment parse is
gumbo_parse_with_options with fragment_context and fragment_namespace
set, so this is a checked convenience wrapper over parse/2 with those
options.
In fragment mode the output's root is a parser-inserted, implied <html>
element — not the context element. The context element lives in gumbo's
internal parser state and is freed with it, so it never appears in the
returned tree; the parsed fragment is the children of that implied root,
which is what Gumbo.Output.fragment_nodes/1 returns.
context sets fragment_context. fragment_namespace defaults to :html;
a non-HTML namespace is how SVG and MathML fragments are requested — :svg
with fragment_namespace: :svg, :math with fragment_namespace: :mathml.
context implies the context element's namespace — :svg implies :svg,
:math implies :mathml, and every other tag implies :html — and an
options entry that disagrees is rejected rather than silently ignored:
fragment_context: in options always conflicts (the positional argument
sets it), and a fragment_namespace: that differs from the implied
namespace returns {:error, {:conflicting_option, key}}. Gumbo.parse/2
performs no such check; it is the unchecked path for combinations this
function refuses.
Examples
iex> {:ok, output} = Gumbo.parse_fragment("<b>x</b>", :div)
iex> Enum.map(Gumbo.Output.fragment_nodes(output), &Gumbo.Node.tag_name/1)
["b"]
iex> Gumbo.parse_fragment("<b>x</b>", :div, fragment_namespace: :svg)
{:error, {:conflicting_option, :fragment_namespace}}
@spec parse_fragment!(binary(), Gumbo.Tag.t(), keyword()) :: Gumbo.Output.t()
Like parse_fragment/3, but raises ArgumentError with the reason on
failure.
Examples
iex> Gumbo.parse_fragment!("<b>x</b>", :div) |> Gumbo.Output.fragment_nodes() |> Enum.map(&Gumbo.Node.tag_name/1)
["b"]