Torque

High-performance JSON library for Elixir via Rustler NIFs, powered by sonic-rs (SIMD-accelerated).

Torque provides the fastest JSON encoding and decoding available in the BEAM ecosystem, with a selective field extraction API for workloads that only need a subset of fields from each document.

Features

• SIMD-accelerated decoding (AVX2 on x86, NEON on ARM)
• Ultra-low memory encoder (64 B per encode vs ~4 KB for OTP json/jason)
• Parse-then-get API for selective field extraction via JSON Pointer (RFC 6901)
• Batch field extraction (get_many/2) with single NIF call
• Pre-compiled pointers with fused parse + extract (parse_get_many_nil/2)
• Automatic dirty CPU scheduler dispatch for decode/parse inputs larger than 20 KB (opt-in dirty: true for encode)
• jiffy-compatible {proplist} encoding

Installation

Add to your mix.exs:

def deps do
  [
    {:torque, "~> 0.2.6"}
  ]
end

Precompiled binaries are available for common targets. To compile from source, install a stable Rust toolchain and set TORQUE_BUILD=true.

CPU-optimized variants

On x86_64, precompiled binaries are available for three CPU feature levels:

At compile time, Torque auto-detects the host CPU and downloads the best matching variant. To override detection (e.g., when cross-compiling for a different target):

TORQUE_CPU_VARIANT=v2 mix compile  # force SSE4.2 variant
TORQUE_CPU_VARIANT=v3 mix compile  # force AVX2 variant
TORQUE_CPU_VARIANT=base mix compile  # force baseline

Usage

Decoding

{:ok, data} = Torque.decode(~s({"name":"Alice","age":30}))
# %{"name" => "Alice", "age" => 30}

data = Torque.decode!(json)

Selective Field Extraction

Parse once, extract many fields without building the full Elixir term tree:

{:ok, doc} = Torque.parse(json)

{:ok, "example.com"} = Torque.get(doc, "/site/domain")
nil = Torque.get(doc, "/missing/field", nil)

# Batch extraction (single NIF call, fastest path)
results = Torque.get_many(doc, ["/id", "/site/domain", "/device/ip"])
# [{:ok, "req-1"}, {:ok, "example.com"}, {:ok, "1.2.3.4"}]

When your JSON is known to have no duplicate object keys, pass unique_keys: true for faster field lookups (uses sonic-rs internal indexing instead of linear scan):

{:ok, doc} = Torque.parse(json, unique_keys: true)

Compiled Pointers

When the same fixed set of paths is extracted from every document, compile the pointers once and reuse the handle. parse_get_many_nil/2 then fuses the parse and extraction into a single NIF call, skipping all per-request path parsing — roughly 1.5× faster end-to-end than parse/2 + get_many_nil/2.

# Once, at startup (e.g. a module attribute or :persistent_term):
pointers = Torque.compile_pointers(["/id", "/site/domain", "/imp/0/banner/w"], unique_keys: true)

# Per document — parse + extract in one call:
{:ok, ["req-1", "example.com", 300]} = Torque.parse_get_many_nil(json, pointers)

Missing fields and JSON null both become nil. The handle also works with an already-parsed document via Torque.get_many_nil(doc, pointers).

Encoding

# Maps with atom or binary keys
{:ok, json} = Torque.encode(%{id: "abc", price: 1.5})
# "{\"id\":\"abc\",\"price\":1.5}"

# Integer keys are stringified — JSON object names must be strings
{:ok, json} = Torque.encode(%{0 => "a", 1 => "b"})
# "{\"0\":\"a\",\"1\":\"b\"}"

# Bang variant
json = Torque.encode!(%{id: "abc"})

# iodata variant (fastest, no {:ok, ...} tuple wrapping)
json = Torque.encode_to_iodata(%{id: "abc"})

# jiffy-compatible proplist format
{:ok, json} = Torque.encode({[{:id, "abc"}, {:price, 1.5}]})

Unlike decoding, encoding cannot cheaply predict its output size, so dirty scheduler dispatch is opt-in. Pass dirty: true (accepted by encode/2, encode!/2, encode_to_iodata/2, and encode_to_iodata!/2) when terms are expected to encode to large output (more than roughly 20 KB):

{:ok, json} = Torque.encode(big_term, dirty: true)

API

Type Conversion

JSON to Elixir

For objects with duplicate keys, the last value wins (unless unique_keys: true is passed to parse/2).

Integers outside the signed/unsigned 64-bit range decode as exact arbitrary-precision integers (Erlang bignums) via decode/1, rather than degrading to lossy floats. The parse/2 + get/2 path returns them as floats, since the parsed document cannot hold a bignum.

Elixir to JSON

Errors

Functions return {:error, reason} tuples (or raise ArgumentError for bang/iodata variants). Possible reason atoms:

Decode / Parse

parse/1, decode/1, and parse_get_many_nil/2 also return {:error, binary} with a message from sonic-rs for malformed JSON.

Encode

Benchmarks

Apple M2 Pro, OTP 29, Elixir 1.20. Both libraries are profile-guided optimised (PGO) builds: Torque PGO (via scripts/pgo-build.sh) and Glazer PGO (via OPTIMIZE=1).

glazer is benchmarked with UTF-8 validation enabled (validate_utf8 on decode, force_utf8 on encode — both off by default in glazer) so every library provides the same guarantee Torque always does: JSON strings are valid UTF-8.

Decode (1.2 KB OpenRTB)

Decode (750 KB Twitter)

Encode (1.2 KB OpenRTB)

Encode (750 KB Twitter)

Parse (1.2 KB OpenRTB)

Extract 5 fields from raw JSON (1.2 KB OpenRTB)

End-to-end cost of pulling 5 fields out of a JSON blob: parse + get (torque) vs decode + find (glazer has no lazy handle, so it must fully decode first). This is the apples-to-apples version of "get" — torque's selective extraction skips materializing the whole document.

Run benchmarks locally:

MIX_ENV=bench mix run bench/torque_bench.exs

Limitations

• Integer map keys are lossy: JSON object names must be strings (RFC 8259 §4), so encode/1 stringifies integer keys and decode/1 gives them back as binaries — %{1 => "a"} round-trips to %{"1" => "a"}. A map mixing both forms, like %{1 => "a", "1" => "b"}, encodes to duplicate names ({"1":"a","1":"b"}); RFC 8259 says names should be unique, and decoders resolve the collision however they choose. Jason behaves identically.
• Nesting depth: JSON documents nested deeper than 128 levels return {:error, :nesting_too_deep} from decode/1, parse/1, get/2, get_many/2, and encode/1 rather than crashing the VM. Real-world documents are never this deep; the limit exists to prevent stack overflow in the NIF (the dirty CPU scheduler, used for inputs over 20 KB, has a small stack).

License

MIT