Laughter (Laughter v0.3.0)

Copy Markdown

A streaming HTML parser for Elixir built on top of the CloudFlare's 😂 LOL HTML.

Summary

Functions

Creates a parser builder.

Creates a parser from a parser builder.

Streams every text chunk in the document, whatever element it is in.

Must be called once you are done parsing.

Selects which elements to stream and where to send them.

Parses a chunk of HTML. Returns the parser for pipelining.

Types

builder_ref()

@type builder_ref() :: reference()

filter_ref()

@type filter_ref() :: non_neg_integer()

parser_ref()

@type parser_ref() :: reference()

Functions

build()

@spec build() :: builder_ref()

Creates a parser builder.

create(builder, opts \\ [])

@spec create(builder_ref(), Keyword.t()) :: parser_ref()

Creates a parser from a parser builder.

Options

  • :encoding - the charset of the file, such as "utf-8". Defaults to "utf-8".
  • :max_memory - maximum allowed size of buffer. Defaults to 16_384.

document_text(builder, pid, opts \\ [])

@spec document_text(builder_ref(), pid(), keyword()) :: filter_ref()

Streams every text chunk in the document, whatever element it is in.

Returns a filter reference; messages are {:text, ref, content} and, at the end, {:end, ref}. Unlike filter/4 with text: true on body, this fires exactly once per chunk and works on documents that omit <body>. Contents of <script>, <style>, and other raw-text elements are skipped unless raw_text: true; <title> and <textarea> text is included.

Examples

ref = Laughter.document_text(builder, self())

done(parser)

@spec done(parser_ref()) :: :ok

Must be called once you are done parsing.

filter(builder, pid, selector, opts \\ false)

@spec filter(builder_ref(), pid(), binary(), boolean() | keyword()) :: filter_ref()

Selects which elements to stream and where to send them.

Returns a filter reference that will be included in messages.

The fourth argument is either the legacy send_content boolean or a keyword list:

  • :text - also send the text inside matched elements as {:text, ref, content}. Defaults to false.
  • :end_tag - send {:end_tag, ref, tag} when a matched element's explicit end tag is seen. Elements closed implicitly (an unclosed <p> followed by another <p>) never produce one. Defaults to false.
  • :raw_text - with :text, also deliver the contents of <script>, <style>, and similar raw-text elements. Defaults to false, so only visible text and the contents of <title> and <textarea> are sent.

Examples

ref = Laughter.filter(builder, self(), ".content > a")
# Messages will be: {:element, ref, {tag, attrs}}

ref = Laughter.filter(builder, self(), "p", text: true, end_tag: true)
# Messages: {:element, ref, {"p", attrs}}, {:text, ref, "..."}, {:end_tag, ref, "p"}

parse(parser, chunk)

@spec parse(parser_ref(), iodata()) :: parser_ref()

Parses a chunk of HTML. Returns the parser for pipelining.