StylesheetParser (StylesheetParser v0.1.0)

Copy Markdown View Source

A CSS Syntax Level 3 tokenizer and parser, with CSS Nesting and a Selectors Level 4 selector parser.

iex> {:ok, stylesheet} = StylesheetParser.parse_stylesheet("a { color: red } .card > b:hover { margin: 0 }")
iex> to_string(stylesheet)
"a { color: red; }\n.card > b:hover { margin: 0; }"

Parsing

parse_stylesheet/1 parses a stylesheet. The others parse a part of one, as the spec's parser entry points do:

FunctionParsesReturns
parse_stylesheet/1a stylesheet{:ok, %StylesheetParser.Stylesheet{}}
parse_rules/1rules, such as an @media block's{:ok, rules, errors}
parse_rule/1one rule{:ok, rule}
parse_declarations/1a style attribute: declarations and nested rules{:ok, contents, errors}
parse_declaration/1one declaration{:ok, %StylesheetParser.Declaration{}}
parse_component_values/1component values, such as a media query list{:ok, values}
parse_component_value/1one component value{:ok, value}
tokenize/1tokens, as StylesheetParser.Syntax.Tokenizer has them{:ok, tokens}

Those that parse one thing are {:error, %StylesheetParser.ParseError{}} given none or more than one, and all of them given input that isn't UTF-8.

iex> {:ok, declarations, []} = StylesheetParser.parse_declarations("color: red; margin: 0 auto")
iex> Enum.map(declarations, &to_string/1)
["color: red;", "margin: 0 auto;"]

iex> StylesheetParser.parse_rule("a { } b { }")
{:error, %StylesheetParser.ParseError{message: "expected one rule", line: 1, column: 7}}

Rules, declarations and component values

A stylesheet's rules are StylesheetParser.QualifiedRules (style rules) and StylesheetParser.AtRules, and a rule's block is its StylesheetParser.Declarations and nested rules. A declaration's value and an at-rule's prelude are component values: StylesheetParser.Functions, StylesheetParser.SimpleBlocks, and the other tokens as {type, value}, each value its source text.

iex> {:ok, declaration} = StylesheetParser.parse_declaration("border: 1px solid var(--line, #ccc)")
iex> declaration.value
[
  {:dimension, ["1", "px"]},
  {:whitespace, " "},
  {:ident, "solid"},
  {:whitespace, " "},
  %StylesheetParser.Function{
    name: "var",
    line: 1,
    column: 19,
    value: [ident: "--line", comma: ",", whitespace: " ", hash: "#ccc"]
  }
]

Rules, declarations, functions and blocks have the line and column where they start.

Malformed CSS

Malformed CSS is dropped, as the spec recovers from it, and each StylesheetParser.ParseError kept:

iex> {:ok, stylesheet} = StylesheetParser.parse_stylesheet("a { color red; margin: 0 }")
iex> to_string(stylesheet)
"a { margin: 0; }"
iex> Enum.map(stylesheet.errors, &Exception.message/1)
["1:5: expected : after color"]

Walking and sanitizing

prewalk/2, postwalk/2 and traverse/4 walk the rules and declarations, to replace or drop them. They don't walk values, so a sanitizer goes through a declaration's value, and an at-rule's prelude, itself. This one replaces each URL with a placeholder: a url() token or a bad url, and a url("...") function, its name compared unescape/1d and downcased as U\52L(...) is one too, wherever they are among the values:

iex> defmodule NoURLs do
...>   alias StylesheetParser.{AtRule, Declaration, Function, SimpleBlock}
...>
...>   @placeholder {:url, "url(about:invalid)"}
...>
...>   def sanitize(stylesheet) do
...>     StylesheetParser.prewalk(stylesheet, fn
...>       %Declaration{value: value} = declaration -> %{declaration | value: values(value)}
...>       %AtRule{prelude: prelude} = rule -> %{rule | prelude: values(prelude)}
...>       rule -> rule
...>     end)
...>   end
...>
...>   defp values(values), do: Enum.map(values, &value/1)
...>
...>   defp value({type, _text}) when type in [:url, :bad_url], do: @placeholder
...>
...>   defp value(%Function{name: name, value: value} = function) do
...>     if url?(name), do: @placeholder, else: %{function | value: values(value)}
...>   end
...>
...>   defp value(%SimpleBlock{value: value} = block), do: %{block | value: values(value)}
...>   defp value(token), do: token
...>
...>   defp url?(name), do: name |> StylesheetParser.unescape() |> String.downcase() == "url"
...> end
iex> {:ok, stylesheet} =
...>   StylesheetParser.parse_stylesheet(
...>     ~S|@import url(theme.css); .hero { background: image-set(url(hero.png) 1x, url("hero@2x.png") 2x), red } .icon { mask: U\52L(icon.svg) }|
...>   )
iex> stylesheet |> NoURLs.sanitize() |> to_string()
"@import url(about:invalid);\n.hero { background: image-set(url(about:invalid) 1x, url(about:invalid) 2x), red; }\n.icon { mask: url(about:invalid); }"

A URL can be a plain string too, as in @import "theme.css" or image-set("hero.png" 1x): to catch those, replace strings where URLs go as well, or allow only the at-rules and functions known to be safe.

Serializing

to_string/1 serializes what they return. Comments are dropped (a /**/ stays only where two tokens would otherwise run together) and whitespace normalized, so the output parses to the same rules again.

Summary

Functions

Used for things that need to consume a single value, like the parsing rules for attr().

Used for the contents of presentational attributes, which parse text into a single declaration’s value, or for parsing a stand-alone selector SELECT or list of Media Queries MEDIAQ, as in Selectors API or the media HTML attribute.

Used in @supports conditions. CSS3-CONDITIONAL

Used for the contents of a style attribute, which parses text into the contents of a single style rule: its declarations, at-rules and nested rules. Returns {:ok, contents, errors}, with the StylesheetParser.ParseErrors for what it dropped or recovered from.

Intended for use by the CSSStyleSheet#insertRule method, and similar functions which might exist, which parse text into a single rule.

Intended for the content of at-rules such as @media. It differs from parse_stylesheet/1 in the handling of <CDO-token> and <CDC-token>. Returns {:ok, rules, errors}, with the StylesheetParser.ParseErrors for what it dropped or recovered from.

Intended to be the normal parser entry point, for parsing stylesheets.

As prewalk/2, calling fun on each rule and declaration after their children.

Calls fun on each rule and declaration of a stylesheet, a list of them, or one of them, before their children (a rule's block), and returns what fun returns in their place: the node, a replacement, or nil to drop it, and then its children aren't walked.

The tokens of css, each {type, value, metadata} like Elixir AST, its metadata [line: l, column: c] where it starts (columns count bytes from 1). Comments are dropped.

Walks the rules and declarations of a stylesheet, a list of them, or one of them, calling pre on each before its children (a rule's block) and post after, with an accumulator. Each returns {node, acc}, the node a replacement or nil to drop it; when pre drops one, its children aren't walked and post isn't called on it. Returns {walked, acc}.

The text of a name, string or url with its escapes decoded, as the spec's token values are: \ and up to six hex digits, and one whitespace after them, is the code point they number (U+FFFD for 0, a surrogate or more than U+10FFFF); \ and a newline is nothing, as in a string that goes on to the next line; \ and any other code point is that code point; and \ at the end is U+FFFD.

Functions

parse_component_value(css)

Used for things that need to consume a single value, like the parsing rules for attr().

parse_component_values(css)

Used for the contents of presentational attributes, which parse text into a single declaration’s value, or for parsing a stand-alone selector SELECT or list of Media Queries MEDIAQ, as in Selectors API or the media HTML attribute.

parse_declaration(css)

Used in @supports conditions. CSS3-CONDITIONAL

parse_declarations(css)

Used for the contents of a style attribute, which parses text into the contents of a single style rule: its declarations, at-rules and nested rules. Returns {:ok, contents, errors}, with the StylesheetParser.ParseErrors for what it dropped or recovered from.

parse_rule(css)

Intended for use by the CSSStyleSheet#insertRule method, and similar functions which might exist, which parse text into a single rule.

parse_rules(css)

Intended for the content of at-rules such as @media. It differs from parse_stylesheet/1 in the handling of <CDO-token> and <CDC-token>. Returns {:ok, rules, errors}, with the StylesheetParser.ParseErrors for what it dropped or recovered from.

parse_stylesheet(css)

Intended to be the normal parser entry point, for parsing stylesheets.

postwalk(node, fun)

As prewalk/2, calling fun on each rule and declaration after their children.

prewalk(node, fun)

Calls fun on each rule and declaration of a stylesheet, a list of them, or one of them, before their children (a rule's block), and returns what fun returns in their place: the node, a replacement, or nil to drop it, and then its children aren't walked.

To sanitize, allow what's known to be safe rather than deny what isn't, and compare names unescape/1d and downcased, as they keep the escapes and case they were written in. Here, keeping only @media at-rules, and declarations without a URL or a function in them:

iex> {:ok, stylesheet} =
...>   StylesheetParser.parse_stylesheet(
...>     ~S|@\69mport "x.css"; a { color: red; b\65havior: url(x.htc); width: \65xpression(alert(1)) } @MEDIA print { b { color: blue; background: u\72l(x.png) } }|
...>   )
iex> sanitized =
...>   StylesheetParser.prewalk(stylesheet, fn
...>     %StylesheetParser.AtRule{name: name} = rule ->
...>       if name |> StylesheetParser.unescape() |> String.downcase() == "@media", do: rule
...>
...>     %StylesheetParser.Declaration{value: value} = declaration ->
...>       if Enum.all?(value, &match?({type, _} when type not in [:url, :bad_url], &1)),
...>         do: declaration
...>
...>     rule ->
...>       rule
...>   end)
iex> to_string(sanitized)
"a { color: red; }\n@MEDIA print { b { color: blue; } }"

tokenize(css)

The tokens of css, each {type, value, metadata} like Elixir AST, its metadata [line: l, column: c] where it starts (columns count bytes from 1). Comments are dropped.

The input is preprocessed first: \r\n, \r and \f become \n, and NUL becomes U+FFFD.

traverse(stylesheet, acc, pre, post)

Walks the rules and declarations of a stylesheet, a list of them, or one of them, calling pre on each before its children (a rule's block) and post after, with an accumulator. Each returns {node, acc}, the node a replacement or nil to drop it; when pre drops one, its children aren't walked and post isn't called on it. Returns {walked, acc}.

iex> {:ok, stylesheet} = StylesheetParser.parse_stylesheet("a { color: red; background: url(x.png) }")
iex> url? = &match?({type, _} when type in [:url, :bad_url], &1)
iex> {sanitized, dropped} =
...>   StylesheetParser.traverse(
...>     stylesheet,
...>     [],
...>     fn
...>       %StylesheetParser.Declaration{value: value} = declaration, dropped ->
...>         if Enum.any?(value, url?),
...>           do: {nil, [declaration | dropped]},
...>           else: {declaration, dropped}
...>
...>       node, dropped ->
...>         {node, dropped}
...>     end,
...>     fn node, dropped -> {node, dropped} end
...>   )
iex> to_string(sanitized)
"a { color: red; }"
iex> Enum.map(dropped, &to_string/1)
["background: url(x.png);"]

unescape(text)

The text of a name, string or url with its escapes decoded, as the spec's token values are: \ and up to six hex digits, and one whitespace after them, is the code point they number (U+FFFD for 0, a surrogate or more than U+10FFFF); \ and a newline is nothing, as in a string that goes on to the next line; \ and any other code point is that code point; and \ at the end is U+FFFD.

Names keep their escapes as written, and a browser reads b\65havior as behavior, so decode names before comparing them.

iex> StylesheetParser.unescape(~S"b\65havior")
"behavior"
iex> StylesheetParser.unescape(~S"@\69 mport")
"@import"
iex> StylesheetParser.unescape(~S"a\:b\31 0")
"a:b10"