Torque

High-performance JSON library for Elixir and Erlang via Rustler NIFs, powered by sonic-rs (SIMD-accelerated).

Torque provides the fastest JSON encoding and decoding available in the BEAM ecosystem, with a selective field extraction API for workloads that only need a subset of fields from each document.

Documentation: hexdocs.pm/torque (Elixir Torque and Erlang torque APIs).

Features

• SIMD-accelerated decoding (AVX2 on x86, NEON on ARM)
• Ultra-low memory encoder (64 B per encode vs ~4 KB for OTP json/jason)
• Parse-then-get API for selective field extraction via JSON Pointer (RFC 6901, with one documented deviation: "/" selects the root, not the empty key)
• Batch field extraction (get_many/2) with single NIF call
• Pre-compiled pointers with fused parse + extract (parse_get_many_nil/2)
• Automatic dirty CPU scheduler dispatch for decode/parse inputs larger than 20 KB (opt-in dirty: true for encode)
• jiffy-compatible {proplist} encoding
• Opt-in Torque.Encoder protocol for encoding structs, with @derive

Installation

Add to your mix.exs:

def deps do
  [
    {:torque, "~> 0.5.1"}
  ]
end

or, with rebar3, to your rebar.config (see Erlang):

{deps, [{torque, "~> 0.5.1"}]}.

Precompiled binaries are available for macOS and glibc Linux on aarch64 and x86_64 (with CPU-optimized variants on x86_64, below). Anything else, such as musl (Alpine) or Windows, builds from source: install a stable Rust toolchain and set TORQUE_BUILD=true.

A source build targets the platform's baseline CPU, so the binary runs on any machine of that architecture. On x86_64 that baseline is SSE2, which leaves the AVX2 parser paths out: set RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq" to match the precompiled v3 variant, or -C target-cpu=native only when the binary will run on the machine that built it.

CPU-optimized variants

On x86_64, precompiled binaries are available for three CPU feature levels:

At compile time, Torque auto-detects the host CPU and downloads the best matching variant, checked against the checksums shipped in the package and cached in the user cache directory. To override detection (e.g., when cross-compiling for a different target):

TORQUE_CPU_VARIANT=v2 mix compile  # force SSE4.2 variant
TORQUE_CPU_VARIANT=v3 mix compile  # force AVX2 variant
TORQUE_CPU_VARIANT=base mix compile  # force baseline

To build on one platform for another (a release built on macOS for Linux, say), set TORQUE_NIF_TARGET to the target's triple, e.g. x86_64-unknown-linux-gnu.

Usage

Decoding

{:ok, data} = Torque.decode(~s({"name":"Alice","age":30}))
# %{"name" => "Alice", "age" => 30}

data = Torque.decode!(json)

Selective Field Extraction

Parse once, extract many fields without building the full Elixir term tree:

{:ok, doc} = Torque.parse(json)

{:ok, "example.com"} = Torque.get(doc, "/site/domain")
nil = Torque.get(doc, "/missing/field", nil)

# Batch extraction (single NIF call, fastest path)
results = Torque.get_many(doc, ["/id", "/site/domain", "/device/ip"])
# [{:ok, "req-1"}, {:ok, "example.com"}, {:ok, "1.2.3.4"}]

When your JSON is known to have no duplicate object keys, pass unique_keys: true for faster field lookups (uses sonic-rs internal indexing instead of linear scan):

{:ok, doc} = Torque.parse(json, unique_keys: true)

Compiled Pointers

When the same fixed set of paths is extracted from every document, compile the pointers once and reuse the handle. parse_get_many_nil/2 then reads the document in a single pass, building values only where a path ends and skipping everything else, without building an intermediate document. Against parse/2 + get_many_nil/2 with the same handle, on PGO builds:

# Once, at startup (e.g. into :persistent_term or application state; the
# handle is a NIF resource, so it cannot live in a module attribute):
pointers = Torque.compile_pointers(["/id", "/site/domain", "/imp/0/banner/w"], unique_keys: true)

# Per document — parse + extract in one call:
{:ok, ["req-1", "example.com", 300]} = Torque.parse_get_many_nil(json, pointers)

Missing fields and JSON null both become nil. The handle also works with an already-parsed document via Torque.get_many_nil(doc, pointers).

By default a malformed document is reported wherever the fault is, as parse/2 would report it, even in a region no path selects. validate: false skips unselected regions with a structural bracket scan instead of tokenizing them, but a malformed number, literal, or separator inside one of them goes unreported, and so does anything after the document, which is therefore not UTF-8 checked either. Truncated input, invalid UTF-8 in any byte the walk consumed, and errors in selected values are still rejected. Use it only with trusted input.

It is not a free speed-up. A bracket scan over 64-byte blocks beats tokenizing a large subtree and loses to it on the few-byte scalars a dense path set leaves behind, so the win tracks how little of the document the paths select. Against a validated handle, on both machines, 3 paths run ~1.9× faster unvalidated on the 1.2 KB request and ~4× faster on the 776 KB feed, while all 51 fields of the request run 3-7% slower. Measure your own path set.

pointers = Torque.compile_pointers(paths, unique_keys: true, validate: false)

Encoding

# Maps with atom or binary keys
{:ok, json} = Torque.encode(%{id: "abc", price: 1.5})
# "{\"id\":\"abc\",\"price\":1.5}"

# Integer keys are stringified — JSON object names must be strings
{:ok, json} = Torque.encode(%{0 => "a", 1 => "b"})
# "{\"0\":\"a\",\"1\":\"b\"}"

# Bang variant
json = Torque.encode!(%{id: "abc"})

# iodata variant (fastest, no {:ok, ...} tuple wrapping)
json = Torque.encode_to_iodata(%{id: "abc"})

# jiffy-compatible proplist format
{:ok, json} = Torque.encode({[{:id, "abc"}, {:price, 1.5}]})

Structs are rejected with {:error, :unhandled_struct} unless they implement Torque.Encoder. Implement the protocol for custom types, or derive it to encode a subset of fields:

defimpl Torque.Encoder, for: Decimal do
  def encode(decimal), do: Decimal.to_string(decimal)
end

# or, on the struct itself:
@derive {Torque.Encoder, only: [:id, :name]}
defstruct [:id, :name, :secret]

Date, Time, NaiveDateTime, and DateTime ship with implementations and encode as ISO 8601 strings.

Breaking change in 0.4.0. Structs previously encoded as raw maps, leaking the struct marker into the output: ~D[2026-09-14] produced {"calendar":"Elixir.Calendar.ISO","month":9,"__struct__":"Elixir.Date",...}. They now error unless the protocol is implemented.

Unlike decoding, encoding cannot cheaply predict its output size, so dirty scheduler dispatch is opt-in. Pass dirty: true (accepted by encode/2, encode!/2, encode_to_iodata/2, and encode_to_iodata!/2) when terms are expected to encode to large output (more than roughly 20 KB):

{:ok, json} = Torque.encode(big_term, dirty: true)

Using with Phoenix, Plug and Postgrex

Torque provides the functions these libraries call on a JSON module (encode_to_iodata!/1, encode!/1, decode!/1), so it can replace Jason in their configuration:

# config/config.exs
config :phoenix, :json_library, Torque
config :postgrex, :json_library, Torque

# endpoint.ex
plug Plug.Parsers,
  parsers: [:urlencoded, :multipart, :json],
  json_decoder: Torque

Structs, Ecto schemas included, must implement Torque.Encoder (for example @derive {Torque.Encoder, only: [:id, :name]}): a struct without an implementation raises rather than encoding its raw fields.

Erlang

The torque module is the Erlang API. It follows Erlang conventions: JSON null is null, and the bulk lookups return undefined for a path the document does not contain, so it stays distinct from a JSON null. Results are {ok, _} | {error, _} tuples and options are proplists.

{ok, #{<<"a">> := null}} = torque:decode(<<"{\"a\":null}">>),
{ok, <<"{\"a\":null}">>} = torque:encode(#{a => null}),

{ok, Doc} = torque:parse(Json),
{ok, Domain} = torque:get(Doc, <<"/site/domain">>),
[Id, undefined] = torque:get_many_values(Doc, [<<"/id">>, <<"/missing">>]),

%% Compile fixed paths once, then extract them in one pass per document:
Pointers = torque:compile_pointers([<<"/id">>, <<"/site/domain">>], [{unique_keys, true}]),
{ok, [Id, Domain]} = torque:parse_get_many_values(Json, Pointers).

The Erlang API guide documents every function and option.

A rebar3 build fetches the precompiled NIF for the platform from the GitHub release, checks it against the checksums shipped in the package, and caches it in the user cache directory. That needs OTP 25 or later; with TORQUE_BUILD=true the NIF is built with cargo instead, which works on any platform. TORQUE_CPU_VARIANT picks the x86_64 variant as it does for Mix.

API

Type Conversion

JSON to Elixir

For objects with duplicate keys, the last value wins (unless unique_keys: true is passed to parse/2).

Integers outside the signed/unsigned 64-bit range decode as exact arbitrary-precision integers (Erlang bignums) rather than degrading to lossy floats, from decode/1 and from every get and parse_get_many_nil lookup alike. One exception: a compile_pointers/2 handle with the default validate: true rejects an integer beyond the f64 range (about 1.8e308) in a region no path selects.

Elixir to JSON

Errors

Functions return {:error, reason} tuples (or raise ArgumentError for bang/iodata variants). Possible reason atoms:

Decode / Parse

parse/1, decode/1, and parse_get_many_nil/2 also return {:error, binary} with a message from sonic-rs for malformed JSON.

Encode

Benchmarks

Per-commit trends and the full cross-library comparison are published at lpgauth.github.io/torque/dev/bench.

Every table below comes from one run of bench/torque_bench.exs, against glazer 1.1.5, on each of two machines:

• arm64: Apple M1 Pro, macOS, OTP 29, Elixir 1.20.3, Apple clang 21, rustc 1.98.1
• x86_64: Intel Xeon E5-2630 v3 (Haswell, 2.4 GHz), Ubuntu 22.04 container pinned to one core of a shared server, OTP 29, Elixir 1.20.2, GCC 13.4, rustc 1.99.0

Both libraries are profile-guided optimised (PGO) builds on both machines: Torque PGO (via scripts/pgo-build.sh, which builds with -C target-cpu=native) and Glazer PGO (via make -C deps/glazer/c_src PGO=generate, the workload in bench/glazer_pgo_workload.exs, then PGO=use). Glazer's Makefile writes that flow for GCC; under clang the raw counters need an explicit llvm-profdata merge -o obj/pgo/default.profdata obj/pgo/*.profraw between those two steps.

glazer is benchmarked with UTF-8 validation enabled (validate_utf8 on decode, force_utf8 on encode — both off by default in glazer) so every library provides the same guarantee Torque always does: JSON strings are valid UTF-8.

Decode (1.2 KB OpenRTB)

arm64

x86_64

Decode (750 KB Twitter)

arm64

x86_64

Encode (1.2 KB OpenRTB)

Function is the call benchmarked. Torque's encode!/1 raises on error and encode_to_iodata/1 returns the same binary without the {:ok, _} tuple; Jason's encode_to_iodata!/1, jiffy and json:encode/1 return iodata. Input is the term encoded: this payload is a bid response with atom keys, the 750 KB one below is decoded JSON, so its keys are binaries.

arm64

x86_64

Encode (750 KB Twitter)

arm64

x86_64

Parse (1.2 KB OpenRTB)

arm64

x86_64

Extract 5 fields from raw JSON (1.2 KB OpenRTB)

End-to-end cost of pulling 5 fields out of a JSON blob: parse + get (torque) vs decode + find (glazer has no lazy handle, so it must fully decode first). This is the apples-to-apples version of "get" — torque's selective extraction skips materializing the whole document.

parse_get_many_nil goes further. Given a handle compiled once at startup (like glazer's compiled jq paths), it walks the document a single time and builds a value only where a path ends, so no document is built at all. validate: false also skips validating the regions no path selects, which on a document this small is most of what is left.

arm64

x86_64

Run benchmarks locally:

MIX_ENV=bench mix run bench/torque_bench.exs

Limitations

• Integer map keys are lossy: JSON object names must be strings (RFC 8259 §4), so encode/1 stringifies integer keys and decode/1 gives them back as binaries — %{1 => "a"} round-trips to %{"1" => "a"}. A map mixing both forms, like %{1 => "a", "1" => "b"}, encodes to duplicate names ({"1":"a","1":"b"}); RFC 8259 says names should be unique, and decoders resolve the collision however they choose. Jason behaves identically.
• Nesting depth: JSON documents nested deeper than 128 levels return {:error, :nesting_too_deep} from decode/1, parse/1, get/2, get_many/2, and encode/1 rather than crashing the VM. Real-world documents are never this deep; the limit exists to prevent stack overflow in the NIF (the dirty CPU scheduler, used for inputs over 20 KB, has a small stack).

License

MIT