Migrating from 0.2 to 0.3

Version 0.3 keeps the existing public function arities and success tuples. It raises the minimum Elixir version and stops encode operations from accepting decode-only labels.

Required: Elixir 1.15 or newer

EncodingRs now requires Elixir 1.15 because Rustler 0.38 and rustler_precompiled 0.9 require it. Applications on Elixir 1.12–1.14 must upgrade Elixir before updating EncodingRs.

def deps do
  [{:encoding_rs, "~> 0.3.0"}]
end

Per-operation options

Scheduler and input limits can now be selected at each call site:

EncodingRs.decode(data, "shift_jis",
  dirty_threshold: 128 * 1024,
  max_input_size: 10 * 1024 * 1024
)

Supported options are:

• :dirty_threshold — non-negative byte count; defaults to 65,536.
• :max_input_size — non-negative byte count or :infinity; defaults to 104,857,600.

Options are validated with Keyword.validate!/2. Unknown keys and invalid values raise ArgumentError instead of being ignored.

Existing config :encoding_rs values remain valid compatibility fallbacks. Explicit options take precedence. :dirty_threshold remains a compile-time fallback and :max_input_size remains a runtime fallback. No deprecation warning is emitted during encoding or decoding.

Batch scheduler behavior changed

In 0.2, batch operations always ran on a dirty CPU scheduler. In 0.3, valid items are totaled and the batch follows :dirty_threshold, just like a one-shot operation. Oversized items are rejected before this total is calculated.

To retain the 0.2 always-dirty behavior:

EncodingRs.decode_batch(items, dirty_threshold: 0)
EncodingRs.encode_batch(items, dirty_threshold: 0)

The batch still crosses the NIF boundary once and preserves input order.

Detailed decoding is additive

decode/2 and decode/3 still return {:ok, string}. Use the new detailed API when replacement or BOM selection must be observable:

{:ok, string, actual_encoding, had_errors} =
  EncodingRs.decode_with_details(data, "windows-1252")

The batch equivalent is decode_batch_with_details/2. Return shapes do not depend on an option such as return_errors: true; callers choose the function whose contract they need.

One-shot decoding has always allowed a leading UTF-8 or UTF-16 BOM to select the actual decoder. Version 0.3 documents that behavior and exposes actual_encoding; it does not change the decode/2 success tuple.

Only the native sentinel for an unrecognized encoding is normalized to :unknown_encoding. Unexpected native error reasons are now preserved instead of being mislabeled. Callers should retain a general {:error, reason} branch.

Stateful decoder contracts

Decoder.new/1 still returns the same opaque reference. Existing decode_chunk/2, decode_chunk/3, decode_chunk!/2, and decode_chunk!/3 calls remain valid. Decoder.new/2 and new!/2 return an opaque configured decoder that stores validated options once. The other new option-taking forms are decode_chunk/4, stream/3, and stream_with_errors/3.

Stream options are resolved once when enumeration starts, then reused for all chunks. For manual loops, prefer new/2 plus decode_chunk/3.

Decoder.decode_chunk/4 may return {:error, :allocation_failed} if its output buffer cannot be allocated. This is in addition to :input_too_large and :lock_poisoned. Code that exhaustively matches decoder errors should add a fallback or handle this atom.

Other API additions and clarifications

• EncodingRs.available?/0 checks whether the native implementation loaded without raising for :nif_not_loaded or :undef.
• Existing application startup behavior is unchanged. The package no longer sets project-only start_permanent metadata.

Decode-only labels now fail encoding

In 0.2, encoding to UTF-16LE, UTF-16BE, or replacement silently produced UTF-8 because those WHATWG encodings have UTF-8 as their output encoding. In 0.3, one-shot and batch encode operations return {:error, :encoder_unavailable} instead. Bang variants raise ArgumentError.

Use a dedicated UTF-16 encoder when UTF-16 output is required.

Upgrade checklist

1. Upgrade to Elixir 1.15 or newer.
2. Change the dependency requirement to ~> 0.3.0 and refresh the lockfile.
3. If batches must always use dirty schedulers, pass dirty_threshold: 0.
4. Audit encode calls that pass UTF-16 or replacement labels.
5. Audit exhaustive matches on stateful decoder errors.
6. Use explicit operation options for reusable libraries; existing application configuration can remain while migrating call sites.