Torque

High-performance JSON library for Elixir via Rustler NIFs, powered by sonic-rs (SIMD-accelerated).

Torque provides among the fastest JSON encoding and decoding available in the BEAM ecosystem, with a selective field extraction API for workloads that only need a subset of fields from each document.

Features

• SIMD-accelerated decoding (AVX2/SSE4.2 on x86, NEON on ARM)
• Ultra-low memory encoder (64 B per encode vs ~4 KB for OTP json/jason)
• Parse-then-get API for selective field extraction via JSON Pointer (RFC 6901)
• Batch field extraction (get_many/2) with single NIF call
• Automatic dirty CPU scheduler dispatch for inputs larger than 20 KB
• jiffy-compatible {proplist} encoding

Installation

Add to your mix.exs:

def deps do
  [
    {:torque, "~> 0.1.10"}
  ]
end

Precompiled binaries are available for common targets. To compile from source, install a stable Rust toolchain and set TORQUE_BUILD=true.

CPU-optimized variants

On x86_64, precompiled binaries are available for three CPU feature levels:

At compile time, Torque auto-detects the host CPU and downloads the best matching variant. To override detection (e.g., when cross-compiling for a different target):

TORQUE_CPU_VARIANT=v2 mix compile  # force SSE4.2 variant
TORQUE_CPU_VARIANT=v3 mix compile  # force AVX2 variant
TORQUE_CPU_VARIANT=base mix compile  # force baseline

Usage

Decoding

{:ok, data} = Torque.decode(~s({"name":"Alice","age":30}))
# %{"name" => "Alice", "age" => 30}

data = Torque.decode!(json)

Selective Field Extraction

Parse once, extract many fields without building the full Elixir term tree:

{:ok, doc} = Torque.parse(json)

{:ok, "example.com"} = Torque.get(doc, "/site/domain")
nil = Torque.get(doc, "/missing/field", nil)

# Batch extraction (single NIF call, fastest path)
results = Torque.get_many(doc, ["/id", "/site/domain", "/device/ip"])
# [{:ok, "req-1"}, {:ok, "example.com"}, {:ok, "1.2.3.4"}]

When your JSON is known to have no duplicate object keys, pass unique_keys: true for faster field lookups (uses sonic-rs internal indexing instead of linear scan):

{:ok, doc} = Torque.parse(json, unique_keys: true)

Encoding

# Maps with atom or binary keys
{:ok, json} = Torque.encode(%{id: "abc", price: 1.5})
# "{\"id\":\"abc\",\"price\":1.5}"

# Bang variant
json = Torque.encode!(%{id: "abc"})

# iodata variant (fastest, no {:ok, ...} tuple wrapping)
json = Torque.encode_to_iodata(%{id: "abc"})

# jiffy-compatible proplist format
{:ok, json} = Torque.encode({[{:id, "abc"}, {:price, 1.5}]})

API

Type Conversion

JSON to Elixir

For objects with duplicate keys, the last value wins (unless unique_keys: true is passed to parse/2).

Elixir to JSON

Errors

Functions return {:error, reason} tuples (or raise ArgumentError for bang/iodata variants). Possible reason atoms:

Decode / Parse

parse/1 and decode/1 also return {:error, binary} with a message from sonic-rs for malformed JSON.

Encode

Benchmarks

Apple M2 Pro, OTP 29, Elixir 1.20:

Decode (1.2 KB OpenRTB)

Decode (750 KB Twitter)

Encode (1.2 KB OpenRTB)

Encode (750 KB Twitter)

Parse (1.2 KB OpenRTB)

Get (5 fields) (1.2 KB OpenRTB)

glazer find runs over a fully decoded term (decode cost excluded, as parse cost is excluded for torque/simdjsone); glazer has no parse-to-handle API, so it is absent from the parse benchmark.

Run benchmarks locally:

MIX_ENV=bench mix run bench/torque_bench.exs

Limitations

• Nesting depth: JSON documents nested deeper than 128 levels return {:error, :nesting_too_deep} from decode/1, get/2, get_many/2, and encode/1 rather than crashing the VM. Real-world documents are never this deep; the limit exists to prevent stack overflow in the NIF (the dirty CPU scheduler, used for inputs over 20 KB, has a small stack).

License

MIT