How a file is punctuated, separately from what its columns mean.
A schema carries the dialect it was declared with. Every read and write accepts the runtime options and applies them on top, so one schema can read the comma-separated export and the tab-separated feed without being declared twice. A call cannot change the layout because the layout determined field positions and embedded shapes when the schema compiled.
delimited_schema :tsv do
field :sku, :string
end
Delimited.read(Product, "supplier.csv", delimiter: ",")Formats
:csv uses a comma. :tsv uses a tab and is otherwise identical, which means
quoting still applies. That is deliberate: the tab-separated files that tools
actually produce quote fields containing tabs, while the IANA
text/tab-separated-values registration forbids such fields outright. Reading
a file that follows the registration works either way; writing produces a file
the registration does not describe only when a value contains a tab, a quote,
or a line break.
:psv uses a pipe and :ssv a single space. A space-separated file is the
one to be most careful with: a single space is a delimiter like any other, so
two spaces make an empty cell between them rather than one wider gap. A file
that aligns its columns with runs of spaces is a fixed-width file, and
layout: :fixed reads it.
:fixed selects the fixed-width layout and turns :headers off, since a
fixed-width file rarely carries a header row.
A format applies exactly the options it names and leaves every other option as
it found it, so merge!(dialect, :tsv) changes the delimiter without
disturbing the null strings.
Layouts
layout: :delimited (the default) finds a cell between two delimiters.
layout: :fixed takes a cell from the byte range each field declares with
:at, and the delimiter and quote character are then unused. See
Delimited.Field for declaring positions.
The layout also decides how records are framed:
record_length: :line(the default) frames a record as a line, however long the line is.record_length: Nframes a record as exactly N bytes, for a file with no line terminators at all. A file that has terminators is framed by its lines whatever its record length, so there is never a question of whether a terminator is part of the record or between two of them.
Options
Reading and writing:
:layout-:delimited(default) or:fixed. The schema declaration selects it; a read or write cannot change it after compilation.:record_length-:line(default) or a positive integer. Fixed layout only.:delimiter- the byte between cells, as a one-character string or a codepoint. Defaults to?,. Delimited layout only.:quote_char- the byte that quotes a cell. Defaults to?". Delimited layout only.:headers- whether the file has a header row. Under the delimited layout,falsematches columns by declaration order. Under the fixed layout, positions always decide, sotrueonly means that the first record is a header line to skip when reading and to write when writing. Defaults totrue, and tofalseunder the:fixedformat.:trim- strip surrounding whitespace from every cell. Defaults tofalse, so that a value is read exactly as the file holds it. Overridable per field.:null- the strings that mean "no value". Defaults to[""]. When writing,nilbecomes the first string in the list. Overridable per field.
Reading only:
:comment- a byte that marks a whole line as a comment, as a one-character string or a codepoint. Defaults tonil, meaning no line is a comment. A commented line is discarded while the file is being framed, before any cell is read, so it may hold anything at all including an unclosed quote.:skip_rows- discard this many rows before the header row. Use it for the export that starts with a title and a blank line. Defaults to0.:skip_blank_lines- ignore lines holding no cells at all. Defaults totrue. A line of only whitespace is not blank; it is a one-cell row.:on_missing_header-:error(default) or:ignore. Ignoring leaves the field at its default for every row, which is why it is not the default: a renamed column would otherwise read as an entire column ofnil.:on_extra_header-:ignore(default) or:error. A schema names the columns it wants, so extra columns are normally the file's business.:chunk_size- bytes read from the file at a time. Defaults to 65_536.
Writing only:
:newline-"\n"(default) or"\r\n". RFC 4180 specifies CRLF; most tools accept either.:quoting-:as_needed(default) quotes a cell only when it holds a delimiter, a quote, or a line break.:alwaysquotes every cell.:bom- write a UTF-8 byte order mark. Defaults tofalse. Use it for a consumer that requires the mark to identify UTF-8. Reading strips a byte order mark either way.:escape_formulas- prefix a cell that a spreadsheet might interpret as a formula with an apostrophe. Defaults tofalse. See the security note below.
Formula escaping
Some spreadsheet import paths interpret a cell beginning with =, +, -,
@, a tab, or a carriage return as a formula. A malicious formula can
disclose data or invoke an external action when the spreadsheet and its
security settings permit one. escape_formulas: true prefixes such a cell
with an apostrophe.
It is off by default because it changes the data: a file written with it and
read back yields '=SUM(A1), not =SUM(A1). Correctness of the round trip
wins over a defence against a hazard in a different program, and the choice is
documented here rather than made silently.
Values that have numeric syntax are not prefixed, so -1.5 survives. This
option is not a universal spreadsheet defence. It checks only the first ASCII
byte after Delimited has chosen the cell boundary. It does not cover a
leading line feed or full-width variants of formula characters. It also does
not protect against a consumer that finds different cell boundaries or a
program that removes the apostrophe when it saves and reopens the file.
OWASP's CSV Injection
guidance records the
same portability limit. Importing the file as text remains the reliable
control.
Summary
Functions
Applies call-site options, or a format name, to an existing dialect.
Builds a dialect from a format name, a keyword list of options, or both.
Types
@type t() :: %Delimited.Dialect{ bom: boolean(), chunk_size: pos_integer(), comment: byte() | nil, delimiter: byte(), escape_formulas: boolean(), headers: boolean(), layout: :delimited | :fixed, newline: String.t(), null: [String.t()], on_extra_header: :error | :ignore, on_missing_header: :error | :ignore, quote_char: byte(), quoting: :as_needed | :always, record_length: :line | pos_integer(), skip_blank_lines: boolean(), skip_rows: non_neg_integer(), trim: boolean() }
Functions
Applies call-site options, or a format name, to an existing dialect.
A format name applies that format's options over the dialect and leaves the
rest of it alone, so merge!(dialect, :tsv) changes the delimiter without
disturbing the null strings.
Raises ArgumentError on an unknown format, or an unknown or invalid option.
Builds a dialect from a format name, a keyword list of options, or both.
Raises ArgumentError for an unknown format, an unknown option, or an option
the dialect cannot honour. A dialect is programmer-owned configuration rather
than data read from a file, so a mistake in one is a mistake in the program.
Delimited.Dialect.new!(:tsv)
Delimited.Dialect.new!(delimiter: ";", newline: "\r\n")
Delimited.Dialect.new!(:fixed, record_length: 100)