Delimit.Reader (delimit v0.1.0)

View Source

Functions for reading delimited data from files or strings.

This module provides functionality to read delimited data from files or strings based on schema definitions, supporting both single-read and streaming operations.

Summary

Types

Options for reading delimited data.

Functions

Reads delimited data from a file.

Reads delimited data from a string.

Streams delimited data from a file.

Types

read_options()

@type read_options() :: [
  headers: boolean(),
  delimiter: String.t(),
  escape: String.t(),
  skip_lines: non_neg_integer(),
  skip_while: (String.t() -> boolean()),
  trim_fields: boolean(),
  nil_on_empty: boolean(),
  format: atom()
]

Options for reading delimited data.

  • :headers - Whether the first row contains headers (default: true)
  • :delimiter - The field delimiter character (default: comma)
  • :escape - The escape character used for quotes (default: double-quote)
  • :skip_lines - Number of lines to skip at the beginning (default: 0)
  • :skip_while - Function that returns true for lines to skip
  • :trim_fields - Whether to trim whitespace from fields (default: true)
  • :nil_on_empty - Convert empty strings to nil (default: true)
  • :format - Predefined format (:csv, :tsv, :psv) that sets appropriate options

Functions

read_file(schema, path, opts \\ [])

@spec read_file(Delimit.Schema.t(), Path.t(), read_options()) :: [struct()]

Reads delimited data from a file.

Parameters

  • schema - The schema definition
  • path - Path to the delimited file
  • opts - Read options that override schema options

Returns

  • List of structs with parsed data based on schema

Examples

iex> MyApp.Person.read("people.csv")
[%MyApp.Person{first_name: "John", last_name: "Doe", age: 42}, ...]

iex> MyApp.Person.read("people.tsv", format: :tsv)
[%MyApp.Person{first_name: "John", last_name: "Doe", age: 42}, ...]

read_string(schema, string, opts \\ [])

@spec read_string(Delimit.Schema.t(), binary(), read_options()) :: [struct()]

Reads delimited data from a string.

Parameters

  • schema - The schema definition
  • string - String containing delimited data
  • opts - Read options that override schema options

Returns

  • List of structs with parsed data based on schema

Examples

iex> csv_data = "first_name,last_name,age\nJohn,Doe,42"
iex> MyApp.Person.read_string(csv_data)
[%MyApp.Person{first_name: "John", last_name: "Doe", age: 42}]

iex> tsv_data = "first_name\tlast_name\tage\nJohn\tDoe\t42"
iex> MyApp.Person.read_string(tsv_data, format: :tsv)
[%MyApp.Person{first_name: "John", last_name: "Doe", age: 42}]

stream_file(schema, path, opts \\ [])

@spec stream_file(Delimit.Schema.t(), Path.t(), read_options()) :: Enumerable.t()

Streams delimited data from a file.

Parameters

  • schema - The schema definition
  • path - Path to the delimited file
  • opts - Read options that override schema options

Returns

  • Stream of structs with parsed data based on schema

Examples

iex> MyApp.Person.stream("large_people_file.csv")
iex> |> Stream.take(10)
iex> |> Enum.to_list()
[%MyApp.Person{first_name: "John", last_name: "Doe", age: 42}, ...]

iex> MyApp.Person.stream("large_people_file.tsv", format: :tsv)
iex> |> Stream.take(10)
iex> |> Enum.to_list()
[%MyApp.Person{first_name: "John", last_name: "Doe", age: 42}, ...]