Jetons.Parser (jetons v0.2.0)

Copy Markdown View Source

Pipeline orchestrator for parsing DTCG config maps into flat token lists.

Coordinates the parsing pipeline:

  1. Jetons.DTCG.apply_extends/1 — resolve $extends inheritance
  2. Jetons.DTCG.flatten/2 — flatten nested maps to {path, value} tuples
  3. Jetons.Ref.expand/1 — expand {ref.path} references
  4. Jetons.Transforms — apply type-based value transforms

Summary

Functions

Extracts unique sorted category prefixes from a token list.

Parses a DTCG config map into a flat list of {path, value} tuples.

Parses multiple groups of JSON files, resolving references across all groups, and returns tokens grouped by label.

parse_tokens(input) deprecated

Functions

extract_categories(tokens)

Extracts unique sorted category prefixes from a token list.

Examples

iex> Jetons.Parser.extract_categories([{"colors.red", "#f00"}, {"spacing.sm", "4px"}, {"colors.blue", "#00f"}])
["colors", "spacing"]

from_config(config)

Parses a DTCG config map into a flat list of {path, value} tuples.

Examples

iex> Jetons.Parser.from_config(%{"colors" => %{"red" => %{"$value" => "#f00"}}})
[{"colors.red", "#f00"}]

from_config(config, opts)

from_files(groups, opts \\ [])

Parses multiple groups of JSON files, resolving references across all groups, and returns tokens grouped by label.

Each group is a {label, files} tuple where files is a list of file paths. References are resolved globally (e.g. tier 2 can reference tier 1 tokens), but the returned map preserves which group each token originated from.

Options

  • :transforms - list of {type, fun} tuples to apply after resolution

Example

Jetons.from_files([
  {"tier1", Path.wildcard("tokens/01-definitions/**/*.json")},
  {"tier2", Path.wildcard("tokens/02-semantics/**/*.json")},
  {"tier3", Path.wildcard("tokens/03-components/**/*.json")}
])
#=> %{"tier1" => [{"color.palette.blue", "#308CE8"}, ...], ...}

get_groups(tokens, category)

See Jetons.Query.get_groups/2.

get_in_group(tokens, category, group)

See Jetons.Query.get_in_group/3.

group_by_path(tokens, category, depth)

See Jetons.Query.group_by_path/3.

parse_tokens(input)

This function is deprecated. Use from_config/1 with explicit file reading instead.