JustBash.Commands.Awk.Lexer (JustBash v0.4.0)

View Source

Lexer for AWK programs.

Tokenizes AWK source code into a stream of tokens that can be consumed by the parser.

Summary

Types

token()
token_type()

Functions

tokenize(input)

Tokenize an AWK program string into a list of tokens.

Types

token()

@type token() ::
  {token_type(), value :: term(), line :: pos_integer(),
   column :: pos_integer()}

token_type()

@type token_type() ::
  :number
  | :string
  | :regex
  | :ident
  | :begin
  | :end
  | :if
  | :else
  | :while
  | :do
  | :for
  | :in
  | :break
  | :continue
  | :next
  | :exit
  | :return
  | :delete
  | :function
  | :print
  | :printf
  | :getline
  | :plus
  | :minus
  | :star
  | :slash
  | :percent
  | :caret
  | :eq
  | :ne
  | :lt
  | :gt
  | :le
  | :ge
  | :match
  | :not_match
  | :and
  | :or
  | :not
  | :assign
  | :plus_assign
  | :minus_assign
  | :star_assign
  | :slash_assign
  | :percent_assign
  | :caret_assign
  | :increment
  | :decrement
  | :question
  | :colon
  | :comma
  | :semicolon
  | :newline
  | :lparen
  | :rparen
  | :lbrace
  | :rbrace
  | :lbracket
  | :rbracket
  | :dollar
  | :append
  | :pipe
  | :eof

Functions

tokenize(input)

@spec tokenize(String.t()) :: {:ok, [token()]} | {:error, String.t()}

Tokenize an AWK program string into a list of tokens.

Returns {:ok, tokens} on success or {:error, message} on failure.