Torque (torque v0.2.4)

Copy Markdown View Source

High-performance JSON library powered by sonic-rs via Rustler NIFs.

Decoding strategies

  • Parse + Getparse/2 returns an opaque document reference. get/2, get/3, get_many/2, and get_many_nil/2 extract fields by JSON Pointer (RFC 6901) paths without materializing the full Elixir term tree. Ideal when only a subset of fields is needed.

  • Compiled pointers — when the same fixed set of paths is extracted from every document, compile_pointers/2 pre-parses the paths once and parse_get_many_nil/2 fuses the parse and extraction into a single NIF call. Skips all per-request path parsing — roughly 1.5× faster end-to-end than parse/2 + get_many_nil/2.

  • Full decodedecode/1 converts an entire JSON binary into Elixir terms in one pass.

Encoding

encode/1 serializes Elixir terms to JSON. Supports maps (atom or binary keys), lists, binaries, numbers, booleans, nil, and jiffy-style {proplist} tuples.

Scheduler awareness

Inputs larger than 20 KB are automatically dispatched to a dirty CPU scheduler to avoid blocking normal BEAM schedulers.

Type conversion

JSONElixir
objectmap with binary keys
arraylist
stringbinary
integerinteger
floatfloat
true / falsetrue / false
nullnil

For objects with duplicate keys, the last value wins (unless unique_keys: true is passed to parse/2).

Summary

Decoding

Decodes a JSON binary into Elixir terms.

Decodes a JSON binary into Elixir terms, raising on error.

Encoding

Encodes an Elixir term into a JSON binary.

Encodes an Elixir term into a JSON binary, raising on error.

Encodes an Elixir term into a JSON binary (iodata-compatible).

Parse + Get

Pre-compiles a list of JSON Pointer paths into a reusable handle.

Extracts a value from a parsed document using a JSON Pointer path (RFC 6901).

Extracts a value from a parsed document, returning default when the path does not exist.

Extracts multiple values from a parsed document in a single NIF call.

Extracts multiple values from a parsed document with per-path defaults.

Extracts multiple values from a parsed document, returning nil for missing fields.

Returns the length of an array at the given JSON Pointer path, or nil if the path does not exist or does not point to an array.

Parses a JSON binary into an opaque document reference.

Parses a JSON binary and extracts pre-compiled pointers in a single NIF call.

Types

An opaque handle to a set of pre-compiled JSON Pointer paths, returned by compile_pointers/2. Pass it to parse_get_many_nil/2 or get_many_nil/2 in place of a path list to skip per-call path parsing.

Decoding

decode(json)

@spec decode(binary()) :: {:ok, term()} | {:error, binary() | :nesting_too_deep}

Decodes a JSON binary into Elixir terms.

JSON objects become maps with binary keys, arrays become lists, strings become binaries, numbers become integers or floats, booleans become true/false, and null becomes nil.

Integers outside the signed/unsigned 64-bit range decode as exact arbitrary-precision integers (Erlang bignums) rather than lossy floats.

Automatically uses a dirty CPU scheduler for inputs larger than 20 KB.

Examples

iex> Torque.decode(~s({"a":1,"b":"hello"}))
{:ok, %{"a" => 1, "b" => "hello"}}

iex> Torque.decode(~s([1,2,3]))
{:ok, [1, 2, 3]}

iex> match?({:error, _}, Torque.decode("invalid"))
true

decode!(json)

@spec decode!(binary()) :: term()

Decodes a JSON binary into Elixir terms, raising on error.

Examples

iex> Torque.decode!(~s({"a":1}))
%{"a" => 1}

Encoding

encode(term)

@spec encode(term()) :: {:ok, binary()} | {:error, binary() | :nesting_too_deep}

Encodes an Elixir term into a JSON binary.

Supported terms

  • Maps with atom or binary keys
  • Lists (JSON arrays)
  • Binaries (JSON strings)
  • Integers and floats
  • true, false, nil (JSON null)
  • Other atoms (encoded as JSON strings)
  • {keyword_list} tuples (jiffy-style proplist objects)

Examples

iex> Torque.encode(%{id: "abc", price: 1.5})
{:ok, ~s({"id":"abc","price":1.5})}

iex> Torque.encode({[{:id, "abc"}]})
{:ok, ~s({"id":"abc"})}

encode!(term)

@spec encode!(term()) :: binary()

Encodes an Elixir term into a JSON binary, raising on error.

Examples

iex> Torque.encode!(%{ok: true})
~s({"ok":true})

encode_to_iodata(term)

@spec encode_to_iodata(term()) :: binary()

Encodes an Elixir term into a JSON binary (iodata-compatible).

Returns the binary directly without {:ok, ...} tuple wrapping. Raises on error. This is the fastest encoding path when the result is passed directly to I/O (e.g. as an HTTP response body).

Examples

iex> Torque.encode_to_iodata(%{ok: true})
~s({"ok":true})

Parse + Get

compile_pointers(paths, opts \\ [])

@spec compile_pointers(
  [binary()],
  keyword()
) :: pointers()

Pre-compiles a list of JSON Pointer paths into a reusable handle.

Workloads that parse many documents and extract the same fixed set of fields re-split and unescape those pointer strings on every call — wasted work, since they never change. compile_pointers/2 does it once and returns an opaque pointers/0 handle that parse_get_many_nil/2 and get_many_nil/2 accept in place of a path list, eliminating all per-call path parsing (≈2× faster extraction on a typical field set).

Compile once at startup (e.g. into a module attribute or :persistent_term) and reuse the handle for every document.

Options

  • :unique_keys — when true, object key lookups use a forward scan that stops at the first match (faster). Defaults to false (reverse scan, last-value-wins for duplicate keys), matching parse/2. Safe to enable when keys are known to be unique.

Extraction results are returned in the same order as paths.

Examples

iex> ptrs = Torque.compile_pointers(["/a", "/b/0"], unique_keys: true)
iex> {:ok, doc} = Torque.parse(~s({"a":1,"b":[2,3]}))
iex> Torque.get_many_nil(doc, ptrs)
[1, 2]

get(doc, path)

@spec get(reference(), binary()) ::
  {:ok, term()} | {:error, :no_such_field | :nesting_too_deep}

Extracts a value from a parsed document using a JSON Pointer path (RFC 6901).

Paths must start with "/". Array elements are addressed by index (e.g. "/imp/0/banner/w"). An empty path "" returns the root value.

Examples

iex> {:ok, doc} = Torque.parse(~s({"site":{"domain":"example.com"}}))
iex> Torque.get(doc, "/site/domain")
{:ok, "example.com"}

iex> {:ok, doc} = Torque.parse(~s({"site":{"domain":"example.com"}}))
iex> Torque.get(doc, "/missing")
{:error, :no_such_field}

get(doc, path, default)

@spec get(reference(), binary(), term()) :: term()

Extracts a value from a parsed document, returning default when the path does not exist.

Raises ArgumentError for errors other than :no_such_field (e.g. :nesting_too_deep).

Examples

iex> {:ok, doc} = Torque.parse(~s({"a":1}))
iex> Torque.get(doc, "/a", nil)
1

iex> {:ok, doc} = Torque.parse(~s({"a":1}))
iex> Torque.get(doc, "/b", :default)
:default

get_many(doc, paths)

@spec get_many(reference(), [binary()]) :: [
  ok: term(),
  error: :no_such_field | :nesting_too_deep
]

Extracts multiple values from a parsed document in a single NIF call.

Returns a list of results in the same order as paths, each being {:ok, value} or {:error, :no_such_field}.

More efficient than calling get/2 in a loop because it crosses the NIF boundary only once.

Examples

iex> {:ok, doc} = Torque.parse(~s({"a":1,"b":2}))
iex> Torque.get_many(doc, ["/a", "/b", "/c"])
[{:ok, 1}, {:ok, 2}, {:error, :no_such_field}]

get_many_defaults(doc, defaults)

@spec get_many_defaults(reference(), %{required(binary()) => term()}) :: %{
  required(binary()) => term()
}

Extracts multiple values from a parsed document with per-path defaults.

Takes a map of %{path => default} and returns a map of the same shape where each value is either the parsed value or the supplied default (if the path is missing).

More ergonomic than the two-call get_many_nil/2 + Enum.map pattern when consumers need defaults at the call site.

Equivalent to:

get_many_nil(doc, Map.keys(defaults))
|> then(&Enum.zip(Map.keys(defaults), &1))
|> Map.new(fn {p, nil} -> {p, Map.get(defaults, p)}; pv -> pv end)

Note: a parsed JSON null at the path is indistinguishable from a missing field (same as get_many_nil/2) — both substitute the default.

Examples

iex> {:ok, doc} = Torque.parse(~s({"a":1,"b":null}))
iex> Torque.get_many_defaults(doc, %{"/a" => 0, "/b" => 0, "/c" => "missing"})
%{"/a" => 1, "/b" => 0, "/c" => "missing"}

get_many_nil(doc, paths)

@spec get_many_nil(reference(), [binary()] | pointers()) :: [term()]

Extracts multiple values from a parsed document, returning nil for missing fields.

Like get_many/2 but returns bare values instead of {:ok, value} tuples. Missing fields return nil (indistinguishable from JSON null).

Faster than get_many/2 when you don't need to distinguish between missing fields and null values, as it avoids allocating wrapper tuples.

Accepts either a list of JSON Pointer path strings or a pointers/0 handle built by compile_pointers/2. The compiled form skips all per-call path parsing and is the recommended option for a fixed, repeatedly-queried path set.

Examples

iex> {:ok, doc} = Torque.parse(~s({"a":1,"b":null}))
iex> Torque.get_many_nil(doc, ["/a", "/b", "/c"])
[1, nil, nil]

iex> {:ok, doc} = Torque.parse(~s({"a":1,"b":null}))
iex> ptrs = Torque.compile_pointers(["/a", "/b", "/c"])
iex> Torque.get_many_nil(doc, ptrs)
[1, nil, nil]

length(doc, path)

@spec length(reference(), binary()) :: non_neg_integer() | nil

Returns the length of an array at the given JSON Pointer path, or nil if the path does not exist or does not point to an array.

Examples

iex> {:ok, doc} = Torque.parse(~s({"a":[1,2,3]}))
iex> Torque.length(doc, "/a")
3

iex> {:ok, doc} = Torque.parse(~s({"a":[1,2,3]}))
iex> Torque.length(doc, "/missing")
nil

parse(json, opts \\ [])

@spec parse(
  binary(),
  keyword()
) :: {:ok, reference()} | {:error, binary()}

Parses a JSON binary into an opaque document reference.

The returned reference can be passed to get/2, get/3, get_many/2, get_many_nil/2, or length/2 for efficient repeated field extraction without re-parsing.

Options

  • :unique_keys — when true, assumes object keys are unique and uses a faster lookup path. Defaults to false (last-value-wins for duplicate keys).

Automatically uses a dirty CPU scheduler for inputs larger than 20 KB.

Examples

iex> {:ok, doc} = Torque.parse(~s({"a":1}))
iex> is_reference(doc)
true

iex> {:ok, doc} = Torque.parse(~s({"a":1}), unique_keys: true)
iex> Torque.get(doc, "/a")
{:ok, 1}

parse_get_many_nil(json, pointers)

@spec parse_get_many_nil(binary(), pointers()) ::
  {:ok, [term()]} | {:error, binary() | :nesting_too_deep}

Parses a JSON binary and extracts pre-compiled pointers in a single NIF call.

Fuses parse/2 and get_many_nil/2 for the common parse-once-extract-once case: it parses the document, extracts each compiled pointer, and returns the values — without materializing a reusable document handle or crossing the NIF boundary twice. Missing fields and JSON null both become nil. The lookup strategy (:unique_keys) is taken from the pointers/0 handle.

Returns {:ok, values} (in the same order as the paths given to compile_pointers/2) or {:error, reason} if the JSON is malformed. Automatically uses a dirty CPU scheduler for inputs larger than 20 KB.

Examples

iex> ptrs = Torque.compile_pointers(["/id", "/site/domain", "/missing"])
iex> Torque.parse_get_many_nil(~s({"id":"x","site":{"domain":"e.com"}}), ptrs)
{:ok, ["x", "e.com", nil]}

iex> ptrs = Torque.compile_pointers(["/a"])
iex> match?({:error, _}, Torque.parse_get_many_nil("not json", ptrs))
true

Types

pointers()

@opaque pointers()

An opaque handle to a set of pre-compiled JSON Pointer paths, returned by compile_pointers/2. Pass it to parse_get_many_nil/2 or get_many_nil/2 in place of a path list to skip per-call path parsing.