ExCellerate (excellerate v0.4.0)

Copy Markdown View Source

ExCellerate is a high-performance expression evaluation engine for Elixir.

It parses text expressions into an intermediate representation (IR), compiles the IR into Elixir AST, and builds a reusable function from it. Compiled functions are cached (keyed by expression and registry), so re-evaluating the same expression skips parsing and compilation.

Operators

  • Arithmetic: +, -, *, /, ^ (power), % (modulo), n! (factorial)
  • Comparison: ==, !=, <, <=, >, >=
  • Logical: &&, ||, not
  • Bitwise: &, |, |^ (xor), <<, >>, ~ (bnot)
  • Ternary: condition ? true_val : false_val
  • Data access: user.profile.name, list[0], list[-1], list[*].field (spread)

Built-in Functions

Math

FunctionDescription
abs(n)Absolute value
round(n) or round(n, digits)Rounds to nearest integer, or to digits decimal places
floor(n)Largest integer ≤ n
ceil(n)Smallest integer ≥ n
trunc(n)Truncates toward zero (unlike floor for negatives)
max(a, b, ...) or max(list)Maximum of arguments or a list
min(a, b, ...) or min(list)Minimum of arguments or a list
sign(n)Returns -1, 0, or 1
sqrt(n)Square root
exp(n)e raised to the power n
ln(n)Natural logarithm (base e)
log(n, base)Logarithm with specified base
log10(n)Base-10 logarithm
sum(a, b, ...) or sum(list)Sums arguments or a list
avg(a, b, ...) or avg(list)Arithmetic mean of arguments or a list

String

FunctionDescription
len(s) or len(list)String length or list length
left(s) or left(s, n)First character, or first n characters
right(s) or right(s, n)Last character, or last n characters
substring(s, start)Substring from start to end
substring(s, start, len)Substring of len characters
upper(s)Converts to uppercase
lower(s)Converts to lowercase
trim(s)Removes leading/trailing whitespace
concat(a, b, ...)Concatenates values into a string
textjoin(delim, a, b, ...)Joins values with a delimiter
replace(s, old, new)Replaces all occurrences of old with new
find(search, text) or find(search, text, start)0-based position of search in text, optionally from start
contains(s, term)Returns true if term exists within s
underscore(s)Downcases, replaces spaces/slashes with underscores, strips special chars
slug(s)Downcases, replaces spaces/slashes with hyphens, strips special chars

Utility

FunctionDescription
if(cond, t) or if(cond, t, f)Returns t if truthy; f (or nil) otherwise
ifs(c1, v1, c2, v2, ...)Returns value for first truthy condition; nil if none match
ifnull(val, default)Returns default if val is nil
isnull(val)Returns true if val is nil, false otherwise
isblank(val)Returns true if val is nil or a whitespace-only string
coalesce(a, b, ...)Returns the first non-nil value
switch(expr, c1, v1, ..., default)Multi-way value matching
and(a, b, ...)Returns true if all arguments are truthy
or(a, b, ...)Returns true if any argument is truthy
lookup(coll, key)Looks up key in a map or index in a list (negative indices count from the end)
lookup(coll, key, default)Same, with a default for missing keys
match(value, list)Returns the 0-based position of value in list (exact match)
match(value, list, type)Approximate match: 1 for ascending (<=), -1 for descending (>=), 0 for exact
index(list, row)Returns the element at 0-based position row (negative indices count from the end)
index(array, row, col)Returns the element at row and col in a 2D array
sort(a, b, ...) or sort(list)Sorts values in ascending order
unique(a, b, ...) or unique(list)Returns unique values, preserving order of first occurrence
filter(list, predicates)Returns items where predicate is true
table(key1, list1, ...)Builds a list of maps from key/list pairs
take(list, rows)Takes the first/last rows elements (negative counts from end)
take(list, rows, cols)Takes rows and columns from a 2D array; pass null to skip a dimension
slice(list, start)Returns elements from start index to end (negative counts from end)
slice(list, start, len)Returns len elements starting at start

Date & Time

FunctionDescription
date(year, month, day)Creates a Date
datetime(year, month, day)Creates a NaiveDateTime at midnight
datetime(year, month, day, hour, minute, second)Creates a NaiveDateTime
today()Returns the current date
now()Returns the current date and time
year(date)Extracts the year
month(date)Extracts the month (1-12)
day(date)Extracts the day of the month (1-31)
hour(date)Extracts the hour (0-23); returns 0 for a Date
minute(date)Extracts the minute (0-59); returns 0 for a Date
second(date)Extracts the second (0-59); returns 0 for a Date
weekday(date)ISO day of the week (1 = Monday, 7 = Sunday)
datedif(date1, date2, unit)Signed difference (date2 − date1) as an integer in the given unit
dateadd(date, amount, unit)Shifts a date by N units; returns the same type

Units for datedif and dateadd (singular or plural): "year(s)", "month(s)", "day(s)", "hour(s)", "minute(s)", "second(s)", "millisecond(s)".

All date functions accept Date, NaiveDateTime, and DateTime structs.

Special Forms

FormDescription
let(name, value, expr)Lexically binds name within expr only

Custom functions can be added via the ExCellerate.Registry system.

Multi-line Expressions

Expressions can be formatted across multiple lines for readability. Newlines are treated as whitespace by the parser:

expr = """
ifs(
  score > 90, 'A',
  score > 80, 'B',
  score > 70, 'C',
  true, 'F'
)
"""

ExCellerate.eval!(expr, %{"score" => 85})
# => "B"

This works with validate/2 and compile/2 as well.

Nil Propagation

Dot and bracket access use nil propagation: if any key in a path is missing or the target is nil, the entire expression returns nil instead of raising an error. This mirrors how spreadsheets treat empty cells and avoids the need for defensive checks at every level of a nested path.

ExCellerate.eval!("user.profile.name", %{"user" => %{}})
# => nil  (profile is missing, so .name is never attempted)

ExCellerate.eval!("user.name", %{"user" => nil})
# => nil  (user is nil, short-circuits)

ExCellerate.eval!("list[99]", %{"list" => [1, 2, 3]})
# => nil  (index out of bounds)

Negative indices count from the end of the list, the same way they work in Elixir:

ExCellerate.eval!("items[-1]", %{"items" => [1, 2, 3]})
# => 3  (last element)

ExCellerate.eval!("items[-2]", %{"items" => [1, 2, 3]})
# => 2  (second to last)

Use ifnull/2, coalesce/2+, or a ternary to provide defaults:

ExCellerate.eval!("ifnull(user.name, 'anonymous')", %{"user" => %{}})
# => "anonymous"

Root variable lookup still raises. Referencing a variable that doesn't exist in the scope at all (e.g., unknown_var) is treated as a likely typo and returns {:error, %ExCellerate.Error{}}:

ExCellerate.eval("totally_unknown", %{})
# => {:error, %ExCellerate.Error{message: "variable not found: totally_unknown"}}

Examples

iex> ExCellerate.eval!("1 + 2 * 3")
7

iex> ExCellerate.eval("a + b", %{"a" => 10, "b" => 20})
{:ok, 30}

iex> ExCellerate.eval!("user.name", %{"user" => %{"name" => "Alice"}})
"Alice"

Column Spread ([*])

The [*] operator extracts a field from every element of a list, returning a new list of values. This enables column-oriented operations on lists of maps, structs, or nested lists:

# Given a list of maps in scope
scope = %{"orders" => [
  %{"product" => "Widget", "price" => 10, "qty" => 2},
  %{"product" => "Gadget", "price" => 25, "qty" => 1}
]}

ExCellerate.eval!("orders[*].product", scope)
# => ["Widget", "Gadget"]

ExCellerate.eval!("sum(orders[*].price)", scope)
# => 35

Spread results work with any function that accepts a list: sum, avg, max, min, len, and textjoin. Subsequent access chains apply to each element:

ExCellerate.eval!("orders[*].price", scope)       # => [10, 25]
ExCellerate.eval!("users[*].profile.name", scope)  # deep access

Bracket indexing after a spread selects a specific position from each element's sub-list:

# scores[1] from each row
ExCellerate.eval!("rows[*].scores[1]", scope)

Nested [*] operators flatten across levels:

 # departments[*].employees[*].name => all employee names, flattened

Computed Spread (.(expr))

To evaluate an expression per element of a spread, use the .(expr) syntax. Inside the parentheses, bare variable names resolve against each element rather than the outer scope:

ExCellerate.eval!("orders[*].(qty * price)", scope)
# => [20, 25]   (per-row product)

ExCellerate.eval!("sum(orders[*].(qty * price))", scope)
# => 45          (total of per-row products)

Computed spreads also compose with nested [*]:

# departments[*].employees[*].(salary * 12)  => annualised salaries, flattened

Let, Filter, and Table

let/3 introduces a lexical binding that is visible only inside the body expression; it does not mutate the outer scope:

# Given scope = %{"orders" => [%{"price" => 10, "qty" => 2}, ...]}
let(total, sum(orders[*].price), total + 5)

filter/2 selects items from a list using a boolean list produced by a computed spread. The predicate list must be the same length as the input list:

filter(orders, orders[*].(qty > 1))

table builds a list of maps from alternating key/list pairs. Use spread or computed spread to produce the list columns:

table('product', orders[*].product, 'total', orders[*].(qty * price))
# => [%{"product" => "Widget", "total" => 20}, ...]

These compose naturally — filter first, then build a summary table:

let(big, filter(orders, orders[*].(qty > 1)),
  table('product', big[*].product, 'total', big[*].(qty * price)))

Take and Slice

take/2 and take/3 extract rows, columns, or both from a list or 2D array. Positive counts take from the beginning, negative from the end. Pass null to skip a dimension:

take(data, 3)          # first 3 rows
take(data, -3)         # last 3 rows
take(data, null, 2)    # first 2 columns (all rows)
take(data, 2, 2)       # first 2 rows and 2 columns

slice/2 and slice/3 extract a contiguous section of a list by start index and optional length. The start index is zero-based, and negative indices count from the end:

slice(items, 1)        # from index 1 to end
slice(items, 1, 3)     # 3 elements starting at index 1
slice(items, -2)       # last 2 elements

Resource Limits

When evaluating untrusted expression strings, two guards bound the work a single expression can trigger. Both are configurable and fall back to safe defaults:

config :excellerate,
  max_expression_length: 10_000,  # max raw input size in bytes
  max_expression_depth: 100,       # max nesting depth of the parsed tree
  max_factorial_input: 10_000      # max operand for the `!` operator

Oversized input is rejected before parsing (a :parser error); expressions nested beyond max_expression_depth are rejected before compilation (a :compiler error); a factorial operand above max_factorial_input is rejected at evaluation (a :runtime error). Raise the limits if your expressions are legitimately large, deeply nested, or need large factorials.

Compilation Strategies

How an expression is executed is a pluggable strategy (ExCellerate.Compilation.Strategy). Two are built in:

  • ExCellerate.Compilation.Interpreted (default) — evaluates via an interpreted Code.eval_quoted/3 closure. No atoms per expression. Safe for any input, and best for expressions evaluated only a few times where native's one-time compile cost would not amortize.
  • ExCellerate.Compilation.NativeCompiled — compiles each expression into a real BEAM module and evaluates it as compiled code: substantially faster on the warm path with much lower per-call allocation. Each distinct compiled expression permanently consumes ~1 atom (an artifact of runtime module creation — the module pool bounds live module memory, not the atom table), so use it for a bounded, trusted set of expressions.

Opt into native compilation globally or per-registry:

config :excellerate, compilation: ExCellerate.Compilation.NativeCompiled

use ExCellerate.Registry, compilation: ExCellerate.Compilation.NativeCompiled

The ExCellerate.NativeCompiler process it needs is started on demand the first time a native compile happens (supervised by the :excellerate application), so there is no supervision wiring and it works whether native is selected globally or only on a single registry. Nothing is started for interpreted-only use.

The ExCellerate.Cache (compiled-function cache) is separate and opt-in — add it to your own supervision tree. See the README for the native_module_limit / native_purge_grace_ms knobs.

Summary

Functions

Compiles an expression into a reusable function.

Similar to compile/2, but returns the function directly or raises on error.

Evaluates a text expression against an optional scope and registry.

Similar to eval/3, but returns the result directly or raises on error.

Validates that an expression is syntactically correct and all referenced functions exist in the registry or defaults.

Types

registry()

@type registry() :: module() | nil

scope()

@type scope() :: %{optional(String.t()) => any()}

Functions

compile(expression, registry \\ nil)

@spec compile(String.t(), registry()) ::
  {:ok, (scope() -> any())} | {:error, ExCellerate.Error.t()}

Compiles an expression into a reusable function.

Returns {:ok, fun} where fun is a 1-arity function that accepts a scope map and returns the result. The compiled function is cached, so calling compile/2 repeatedly with the same expression is cheap.

This is useful when you need to evaluate the same expression many times with different scopes — the parsing and compilation happen only once.

Examples

iex> {:ok, fun} = ExCellerate.compile("a + b")
iex> fun.(%{"a" => 1, "b" => 2})
3

compile!(expression, registry \\ nil)

@spec compile!(String.t(), registry()) :: (scope() -> any())

Similar to compile/2, but returns the function directly or raises on error.

Examples

iex> fun = ExCellerate.compile!("x * 2")
iex> fun.(%{"x" => 5})
10

eval(expression, scope \\ %{}, registry \\ nil)

@spec eval(String.t(), scope(), registry()) :: {:ok, any()} | {:error, Exception.t()}

Evaluates a text expression against an optional scope and registry.

Returns {:ok, result} on success or {:error, %ExCellerate.Error{}} on failure.

Parameters

  • expression — a string containing the expression.
  • scope — a map of variables available to the expression. Supports string keys, atom keys, and structs. Defaults to %{}.
  • registry — an optional module created with use ExCellerate.Registry.

Examples

iex> ExCellerate.eval("1 + 2 * 3")
{:ok, 7}

iex> ExCellerate.eval("a + b", %{"a" => 10, "b" => 20})
{:ok, 30}

iex> ExCellerate.eval("user.name", %{"user" => %{"name" => "Alice"}})
{:ok, "Alice"}

iex> {:error, _} = ExCellerate.eval("1 + * 2")

eval!(expression, scope \\ %{}, registry \\ nil)

@spec eval!(String.t(), scope(), registry()) :: any()

Similar to eval/3, but returns the result directly or raises on error.

Examples

iex> ExCellerate.eval!("1 + 2 * 3")
7

iex> ExCellerate.eval!("a > 10 ? 'high' : 'low'", %{"a" => 15})
"high"

iex> ExCellerate.eval!("concat('Hello', ' ', name)", %{"name" => "Alice"})
"Hello Alice"

iex> ExCellerate.eval!("user.profile.id", %{"user" => %{"profile" => %{"id" => 1}}})
1

validate(expression, registry \\ nil)

@spec validate(String.t(), registry()) :: :ok | {:error, ExCellerate.Error.t()}

Validates that an expression is syntactically correct and all referenced functions exist in the registry or defaults.

Returns :ok if the expression is valid, or {:error, %ExCellerate.Error{}} if it contains syntax errors or references unknown functions.

Note: Validation checks syntax, function existence, and arity, but does not perform type checking. Since scope values are not known until runtime, type errors (e.g., passing a number to upper/1) will only be caught at evaluation time.

Examples

iex> ExCellerate.validate("1 + 2")
:ok

iex> ExCellerate.validate("abs(-1)")
:ok

iex> {:error, _} = ExCellerate.validate("1 +")

iex> {:error, _} = ExCellerate.validate("unknown_func(1)")