Gumbo (gumbo v0.3.0)

Copy Markdown View Source

HTML parsing on top of gumbo-parser 0.14.1.

parse/2 and parse_fragment/3 return an opaque Gumbo.Output handle backed by a NIF resource: the tree stays native and is walked lazily through Gumbo.Node accessors. The handle holds a private copy of the input and is freed when it is garbage collected.

Summary

Functions

Converts a handle, a node, a list of nodes, or nil into plain nested Elixir data in a single NIF call.

Parses html and returns a Gumbo.Output handle.

Like parse/2, but raises ArgumentError with the reason on failure.

Parses html as a fragment whose context element is context, returning a Gumbo.Output handle.

Functions

materialize(value, options \\ [])

@spec materialize(
  Gumbo.Output.t() | Gumbo.Node.t() | [Gumbo.Node.t()] | nil,
  keyword()
) :: term()

Converts a handle, a node, a list of nodes, or nil into plain nested Elixir data in a single NIF call.

The walk is iterative in C — the node table is preorder, and each node's map is built from its already-built children in one reverse pass — so depth is bounded by memory, not by the scheduler's C stack.

Gumbo.Output.t() materializes to %{document: ..., root: ..., errors: [...]}, Gumbo.Node.t() to that node's map, a list to the list of its nodes' maps, and nil to nil. root is nil for a stop_on_first_error parse that halted before any element was built — that handle has no root, only errors.

detail: :full (the default) carries every gumbo field, with nil where the node type has none; detail: :tree returns the tree a caller processes rather than queries — each element as %{type:, tag_name:, attributes:, children:}, with the name as written and the attributes in source order as {name, value} pairs whose prefix the parser split off is composed back on — and every other node as :content has it; detail: :content returns a usable tree without the source bookkeeping (positions, originals, flags and index) — except that an element whose tag is :unknown also carries tag_name, because its atom is the one thing that cannot name it, and a processing instruction also carries target and data, the two parts the spec gives that payload. The materialized :full values agree with the lazy accessors — they come from the same extractors — under the materialized shape's own key names (start_position, name).

detail: is validated here and raises ArgumentError for anything but :full | :tree | :content.

Like parse/2, materializations of 20 480 nodes or more run on a dirty CPU scheduler — the size proxy is the handle's cached node count, so the dispatch costs no NIF call — with the same dirty: override; it raises ArgumentError for anything but :auto | true | false.

Examples

iex> {:ok, output} = Gumbo.parse("<!doctype html><p>x</p>")
iex> Gumbo.materialize(output) |> Map.keys() |> Enum.sort()
[:document, :errors, :root]

iex> {:ok, output} = Gumbo.parse("<!doctype html><p>x</p>")
iex> Gumbo.materialize(output, detail: :content)[:root][:tag]
:html

iex> {:ok, output} = Gumbo.parse_fragment(~s(<p class="a" id="b">x</p>), :body)
iex> [p] = Gumbo.Output.fragment_nodes(output)
iex> Gumbo.materialize(p, detail: :tree)[:attributes]
[{"class", "a"}, {"id", "b"}]

parse(html, options \\ [])

@spec parse(binary(), keyword()) :: {:ok, Gumbo.Output.t()} | {:error, term()}

Parses html and returns a Gumbo.Output handle.

options are validated by Gumbo.Options.new/1; an unknown key or an out-of-range value returns {:error, {:invalid_option, key, value}}. Inputs over 16 MiB are rejected with {:error, :input_too_large}: the parse recurses on the input's nesting depth, so the C stack is sized from the input's length, and that limit is where the reservation stays reasonable. An element with more than :max_attributes attribute names (1024 by default, a repeated name included) is rejected with {:error, :too_many_attributes} rather than having the extra attributes dropped, because a tree missing attributes would be one this library's fidelity claim does not cover. A document past :max_tree_depth open elements (no limit by default) is rejected with {:error, :tree_too_deep} rather than handed back truncated, for the same reason. A document that runs the parse out of the C stack it was given is rejected with {:error, :too_deep}: the stack is sized from the input's length, so this is a backstop under the arithmetic rather than a limit a caller has to think about.

Parses of 20 KiB or more (Gumbo.Options.dirty_threshold/0 bytes) run on a dirty CPU scheduler so a large document does not block a normal scheduler; dirty: false forces the normal scheduler, dirty: true the dirty one. On a VM without dirty schedulers the dirty choice falls back to the normal path invisibly.

Examples

iex> {:ok, output} = Gumbo.parse("<!doctype html><p>x</p>")
iex> Gumbo.Node.type(Gumbo.Output.root(output))
:element

parse!(html, options \\ [])

@spec parse!(binary(), keyword()) :: Gumbo.Output.t()

Like parse/2, but raises ArgumentError with the reason on failure.

Examples

iex> Gumbo.parse!("<!doctype html><p>x</p>") |> Gumbo.Output.node_count()
6

parse_fragment(html, context, options \\ [])

@spec parse_fragment(binary(), Gumbo.Tag.t(), keyword()) ::
  {:ok, Gumbo.Output.t()} | {:error, term()}

Parses html as a fragment whose context element is context, returning a Gumbo.Output handle.

Gumbo has no gumbo_parse_fragment: a fragment parse is gumbo_parse_with_options with fragment_context and fragment_namespace set, so this is a checked convenience wrapper over parse/2 with those options.

In fragment mode the output's root is a parser-inserted, implied <html> element — not the context element. The context element lives in gumbo's internal parser state and is freed with it, so it never appears in the returned tree; the parsed fragment is the children of that implied root, which is what Gumbo.Output.fragment_nodes/1 returns.

context sets fragment_context. fragment_namespace defaults to :html; a non-HTML namespace is how SVG and MathML fragments are requested — :svg with fragment_namespace: :svg, :math with fragment_namespace: :mathml.

context implies the context element's namespace — :svg implies :svg, :math implies :mathml, and every other tag implies :html — and an options entry that disagrees is rejected rather than silently ignored: fragment_context: in options always conflicts (the positional argument sets it), and a fragment_namespace: that differs from the implied namespace returns {:error, {:conflicting_option, key}}. Gumbo.parse/2 performs no such check; it is the unchecked path for combinations this function refuses.

Examples

iex> {:ok, output} = Gumbo.parse_fragment("<b>x</b>", :div)
iex> Enum.map(Gumbo.Output.fragment_nodes(output), &Gumbo.Node.tag_name/1)
["b"]

iex> Gumbo.parse_fragment("<b>x</b>", :div, fragment_namespace: :svg)
{:error, {:conflicting_option, :fragment_namespace}}

parse_fragment!(html, context, options \\ [])

@spec parse_fragment!(binary(), Gumbo.Tag.t(), keyword()) :: Gumbo.Output.t()

Like parse_fragment/3, but raises ArgumentError with the reason on failure.

Examples

iex> Gumbo.parse_fragment!("<b>x</b>", :div) |> Gumbo.Output.fragment_nodes() |> Enum.map(&Gumbo.Node.tag_name/1)
["b"]