Delimited.Type behaviour (Delimited v0.4.0)

Copy Markdown View Source

The built-in field types and the behaviour for defining your own.

A type is the only place where the text in a file becomes an Elixir term. Reading calls cast/2 with the cell's text. Writing calls dump/2 with the term, then applies the declared read rules and calls cast/2 on the resulting text. The writer refuses the value unless casting returns the same term. Neither callback receives nil: an empty cell becomes the field's default without reaching the type, and a nil value uses the field's null handling.

Built-in types

TypeReadsWrites
:stringthe cell unchangedthe binary unchanged
:integeran optionally signed decimal integerInteger.to_string/1
:floatan integer or decimal, with an optional exponentFloat.to_string/1
:booleantrue/t/yes/y/1 and false/f/no/n/0, any case"true" or "false"
:dateDate.from_iso8601/1Date.to_iso8601/1
:timeTime.from_iso8601/1Time.to_iso8601/1
:naive_datetimeNaiveDateTime.from_iso8601/1NaiveDateTime.to_iso8601/1
:utc_datetimeDateTime.from_iso8601/1, converted to UTCDateTime.to_iso8601/1
:decimalDecimal.parse/1, finite values onlyDecimal.to_string/2 in :normal form
{:enum, values}one of the declared stringsthe string declared for the term

The whole cell must be consumed. "12abc" is not an integer and "1.0" is not an integer, because a partial read is how silently wrong numbers enter a data set.

:utc_datetime accepts any ISO 8601 offset and converts to UTC, so 2024-03-01T12:00:00+02:00 reads as 2024-03-01 10:00:00Z. Writing it back produces the UTC form, not the original offset.

:decimal requires the optional :decimal dependency, at any 2.x or 3.x version. Declaring the type without it raises at compile time.

Which of those you resolve decides what happens to a hostile number, so it is worth choosing rather than inheriting. Decimal 3.0 made the IEEE 754 decimal128 limits its defaults, mitigating CVE-2026-32686: a cell holding 1e1000000000 is refused as it is read. Every 2.x version accepts that cell, and writing the value back renders it in full, at a length that grows with the exponent, which a file you do not control can therefore use to exhaust memory. Require {:decimal, "~> 3.0"} if you read files from anywhere you do not trust.

No built-in type parses a thousands separator or a currency symbol. Define a type for those.

Dates and times that are not ISO 8601

A file writing 01/03/2024 needs to say which way round it is, and :format is where it says so:

field :invoiced_on, :date, format: "%d/%m/%Y"
field :due_on, :date, format: "%m/%d/%Y"

The directives are Calendar.strftime/3's own, so one declaration serves both directions: reading uses it, and writing hands it to the standard library. Only those that can be read back are accepted, and a format is checked when the schema compiles rather than on the first file.

DirectiveReads
%Ya year of up to four digits
%ya two-digit year. See the century note below
%ma month number
%da day
%H, %M, %Shour, minute, second
%Ba month's full English name, in any case
%ba month's abbreviated English name, in any case
%%a literal %

Any other character in the format matches itself, so %d-%b-%Y reads 01-Mar-2024. A number is read greedily up to its width, which means a format writing 01 still reads 1 where a separator follows it, and that a format with no separators at all, such as %Y%m%d, still divides correctly.

Declaring several formats reads a supplier who cannot keep to one:

field :invoiced_on, :date, format: ["%d/%m/%Y", "%Y-%m-%d"]

They are tried in order, and the first is the one written. A declared format replaces ISO 8601 rather than adding to it, so a field declared "%d/%m/%Y" refuses 2024-03-01. Accepting both would let one file carry two spellings of the same date and be read without complaint.

%y has to guess a century the file never stated. It uses the POSIX window, where 69 to 99 are the 1900s and 00 to 68 are the 2000s, so 15-Jun-99 reads as 1999 and 15-Jun-00 as 2000. Where a file's own convention differs, and for anything meant to outlive 2068, a four-digit year is the only honest fix.

A format must state everything its type needs. format: "%Y-%m" on a :date is refused when the schema compiles, because the alternative is every date silently landing on the first of the month.

A :utc_datetime read through a format is taken as UTC, since a format of this kind carries no offset. Leave the type on ISO 8601 where the file states one.

Enumerations

{:enum, [:draft, :published]} reads and writes the atom's own text. {:enum, [draft: "D", published: "P"]} maps each atom to the text the file uses. Any other text in the cell is a :cast_failed error, which is the point of declaring the enumeration.

Defining a type

defmodule Postcode do
  @behaviour Delimited.Type

  @impl true
  def cast(text, _opts) do
    case Regex.run(~r/^([A-Z]{1,2}\d[A-Z\d]?) ?(\d[A-Z]{2})$/, String.upcase(text)) do
      [_, outward, inward] -> {:ok, outward <> " " <> inward}
      nil -> {:error, "a UK postcode"}
    end
  end

  @impl true
  def dump(postcode, _opts) when is_binary(postcode), do: {:ok, postcode}
  def dump(_other, _opts), do: {:error, "a UK postcode"}
end

Use it as field :postcode, Postcode. The error string completes the sentence "cannot read value as ...", so write a noun phrase rather than a sentence.

cast/2 and dump/2 must be inverses for every value the type writes. The writer checks that contract before it emits a field and returns :unrepresentable_value when the callbacks disagree.

Options that Delimited.Field does not recognise are passed to a custom type as opts, so field :price, Money, currency: "GBP" reaches cast/2 as [currency: "GBP"]. Built-in types accept no options and reject unknown ones, because there a stray key is a typo rather than configuration.

Summary

Types

What the type expected, as a noun phrase completing "cannot read X as ...".

t()

A built-in type name, an enumeration, or a module implementing this behaviour.

Callbacks

Converts one cell's text into a term.

Converts a term into the text for one cell.

Functions

Returns the built-in type names.

Reads one cell's text as type.

Describes what a type accepts, as a noun phrase.

Writes a term as one cell's text.

Checks a declared type and returns it in the form the reader and writer use.

Types

expectation()

@type expectation() :: String.t()

What the type expected, as a noun phrase completing "cannot read X as ...".

t()

@type t() ::
  :string
  | :integer
  | :float
  | :boolean
  | :date
  | :time
  | :naive_datetime
  | :utc_datetime
  | :decimal
  | {:enum, [{atom(), String.t()}]}
  | module()

A built-in type name, an enumeration, or a module implementing this behaviour.

Callbacks

cast(text, opts)

@callback cast(text :: String.t(), opts :: keyword()) ::
  {:ok, term()} | {:error, expectation()}

Converts one cell's text into a term.

Never called with nil. Return {:error, expectation} where the expectation completes the sentence "cannot read value as ...".

dump(value, opts)

@callback dump(value :: term(), opts :: keyword()) ::
  {:ok, iodata()} | {:error, expectation()}

Converts a term into the text for one cell.

Never called with nil. Returning iodata avoids a copy for composite values.

Functions

builtins()

@spec builtins() :: [atom()]

Returns the built-in type names.

cast(module, text, opts)

@spec cast(t(), String.t(), keyword()) :: {:ok, term()} | {:error, expectation()}

Reads one cell's text as type.

describe(module)

@spec describe(t()) :: expectation()

Describes what a type accepts, as a noun phrase.

dump(module, value, opts)

@spec dump(t(), term(), keyword()) :: {:ok, iodata()} | {:error, expectation()}

Writes a term as one cell's text.

validate!(type)

@spec validate!(term()) :: t()

Checks a declared type and returns it in the form the reader and writer use.

Enumerations are normalised to a keyword list of term-to-text pairs, so {:enum, [:a]} becomes {:enum, [a: "a"]}. Every other type is returned unchanged.

Raises ArgumentError for an unknown type. Called at compile time by the schema DSL, so a typo fails the build rather than the first read.