RustyCSV is an ultra-fast CSV parsing and dumping library powered by purpose-built Rust NIFs.

It provides a drop-in replacement for NimbleCSV with the same API, while offering multiple parsing strategies optimized for different use cases.

Quick Start

Use the pre-defined RustyCSV.RFC4180 parser:

alias RustyCSV.RFC4180, as: CSV

CSV.parse_string("name,age\njohn,27\n")
#=> [["john", "27"]]

CSV.parse_string("name,age\njohn,27\n", skip_headers: false)
#=> [["name", "age"], ["john", "27"]]

Defining Custom Parsers

You can define custom CSV parsers with define/2:

RustyCSV.define(MyParser,
  separator: ",",
  escape: "\"",
  line_separator: "\n"
)

MyParser.parse_string("a,b\n1,2\n")
#=> [["1", "2"]]

Parsing Strategies

RustyCSV supports the :strategy option for backward compatibility, but :simd, :basic, :indexed, and :zero_copy are all equivalent — they use the same SIMD structural boundary scanner and hybrid sub-binary term builder. The only meaningfully distinct strategies are:

• :simd (default) — SIMD structural boundary scan, single-threaded
• :parallel — same SIMD scan, multi-threaded field extraction via rayon (best for very large files 500 MB+)

The streaming parser (used automatically with parse_stream/2) is a separate stateful approach for bounded-memory processing.

Example:

CSV.parse_string(large_csv, strategy: :parallel)

Scheduling

All parsing NIFs run on BEAM dirty CPU schedulers, so they never block normal schedulers. Parallel parsing (:parallel strategy) additionally runs on a dedicated rustycsv-* rayon thread pool to avoid contention with other Rayon users in the same VM.

Streaming

For large files, use parse_stream/2 which uses a bounded-memory streaming parser:

"huge.csv"
|> File.stream!()
|> CSV.parse_stream()
|> Stream.each(&process_row/1)
|> Stream.run()

Streaming parsers are safe to share across processes — the underlying Rust resource is protected by a mutex. However, concurrent access is serialized, so for maximum throughput use one parser per process.

Encoding (Dumping)

Convert rows back to CSV format:

CSV.dump_to_iodata([["name", "age"], ["john", "27"]])
#=> [["name,age\n"], ["john,27\n"]]

Encoding uses SIMD-accelerated Rust NIFs and returns one top-level iodata list per row, matching NimbleCSV's public return shape. Use IO.iodata_to_binary/1 when a flat binary is required. The NIF handles plain UTF-8, formula escaping, non-UTF-8 encoding, and combinations of those modes.

Encoding Strategies

dump_to_iodata/2 accepts a :strategy option:

• default (no option) — Single-threaded SIMD-accelerated encoder. Returns one binary per row. Best for most workloads.

• :parallel — Multi-threaded encoding via rayon. Copies all field data into Rust-owned memory and encodes rows across worker threads. Returns one binary per row. Best for quoting-heavy data (user-generated content with embedded commas/quotes/newlines).

Example:

# Default (recommended for most cases)
CSV.dump_to_iodata(rows)

# Parallel (opt in for quoting-heavy data)
CSV.dump_to_iodata(rows, strategy: :parallel)

High-Throughput Concurrent Exports

The encoding NIF runs on dirty CPU schedulers with per-thread mimalloc arenas, making it suitable for concurrent export workloads — e.g., thousands of users downloading CSV reports simultaneously:

# Phoenix controller — each request encodes independently
rows = MyApp.Reports.fetch_rows(user_id)
csv = MyCSV.dump_to_iodata(rows)
send_download(conn, {:binary, csv}, filename: "report.csv")

For very large exports, use chunked NIF encoding for bounded memory:

MyApp.Reports.stream_rows(user_id)
|> Stream.chunk_every(5_000)
|> Stream.map(&MyCSV.dump_to_iodata/1)
|> Enum.each(&Conn.chunk(conn, &1))

NimbleCSV Compatibility

RustyCSV is designed as a drop-in replacement for NimbleCSV. The API is identical:

• parse_string/2 - Parse CSV string to list of rows
• parse_stream/2 - Lazily parse a stream
• parse_enumerable/2 - Parse any enumerable
• dump_to_iodata/2 - Convert rows to list-shaped iodata
• dump_to_stream/1 - Lazily convert rows to iodata stream
• to_line_stream/1 - Convert arbitrary chunks to lines
• options/0 - Return the original definition options

RustyCSV extends NimbleCSV with additional options:

• :strategy on parse_string/2 - Select the parsing approach (:simd, :basic, :indexed, :parallel, :zero_copy)
• :strategy on dump_to_iodata/2 - Select the encoding approach (default or :parallel)
• :headers - Return rows as maps instead of lists

Headers-to-Maps

Use the :headers option to get maps instead of lists:

CSV.parse_string("name,age\njohn,27\n", headers: true)
#=> [%{"name" => "john", "age" => "27"}]

CSV.parse_string("name,age\njohn,27\n", headers: [:name, :age])
#=> [%{name: "john", age: "27"}]

CSV.parse_string("name,age\njohn,27\n", headers: ["n", "a"])
#=> [%{"n" => "john", "a" => "27"}]

Streaming also supports headers:

"huge.csv"
|> File.stream!()
|> CSV.parse_stream(headers: true)
|> Stream.each(&process_map/1)
|> Stream.run()

How :headers interacts with :skip_headers

With headers: true, the first row is always consumed as keys — :skip_headers has no effect.

With headers: [keys], the :skip_headers option controls whether the first row is skipped (default: true). Most CSV files have a header row, so skipping it avoids mapping the header row itself into a map. If your file has no header row, pass skip_headers: false:

# File with header row (typical) — first row skipped by default
CSV.parse_string("name,age\njohn,27\n", headers: [:n, :a])
#=> [%{n: "john", a: "27"}]

# File without header row — include all rows
CSV.parse_string("john,27\njane,30\n", headers: [:n, :a], skip_headers: false)
#=> [%{n: "john", a: "27"}, %{n: "jane", a: "30"}]

Edge cases

• Fewer columns than keys — missing values are nil
• More columns than keys — extra columns are ignored
• Duplicate headers — last column wins
• Empty header field — key is ""

Multi-Separator Support

Like NimbleCSV, RustyCSV supports multiple separator characters. Separators can be single-byte or multi-byte:

RustyCSV.define(MyParser,
  separator: [",", ";"],
  escape: "\""
)

# Any separator in the list is recognized when parsing
MyParser.parse_string("a,b;c\\n1;2,3\\n", skip_headers: false)
#=> [["a", "b", "c"], ["1", "2", "3"]]

# Only the FIRST separator is used when dumping
MyParser.dump_to_iodata([["a", "b", "c"]]) |> IO.iodata_to_binary()
#=> "a,b,c\\n"

Multi-byte separators are supported:

RustyCSV.define(MyParser,
  separator: "::",
  escape: "\""
)

MyParser.parse_string("a::b::c\\n", skip_headers: false)
#=> [["a", "b", "c"]]

You can also mix single-byte and multi-byte separators:

RustyCSV.define(MyParser,
  separator: [",", "::"],
  escape: "\""
)

Multi-Byte Escape Support

Escape sequences can also be multi-byte:

RustyCSV.define(MyParser,
  separator: ",",
  escape: "$$"
)

MyParser.parse_string("$$hello$$,world\\n", skip_headers: false)
#=> [["hello", "world"]]

Encoding Support

RustyCSV supports character encoding conversion via the :encoding option. This is useful when exporting CSVs with non-ASCII characters (accents, CJK, emoji) that need to open correctly in spreadsheet applications:

alias RustyCSV.Spreadsheet

# Export data with international characters for Excel/Google Sheets/Numbers
rows = [["名前", "年齢"], ["田中", "27"], ["Müller", "35"]]
csv = Spreadsheet.dump_to_iodata(rows) |> IO.iodata_to_binary()
File.write!("export.csv", csv)

The pre-defined RustyCSV.Spreadsheet module outputs UTF-16 LE with BOM, which spreadsheet applications auto-detect correctly. You can also define custom encodings:

RustyCSV.define(MySpreadsheet,
  separator: "\t",
  encoding: {:utf16, :little},
  trim_bom: true,
  dump_bom: true
)

Supported encodings:

• :utf8 - UTF-8 (default, no conversion overhead)
• :latin1 - ISO-8859-1 / Latin-1
• {:utf16, :little} - UTF-16 Little Endian
• {:utf16, :big} - UTF-16 Big Endian
• {:utf32, :little} - UTF-32 Little Endian
• {:utf32, :big} - UTF-32 Big Endian