hnc_csv (hnc_csv v0.2.0)

View Source

CSV Decoder/Encoder

Summary

Functions

Decodes the raw CSV document given in RawData, using the given Options, into a CSV data structure.

Adds another chunk of unprocessed RawData to the given decoder State.

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as argument.

Combines the functionality of decode_filter and decode_map.

Flushes a possibly unfinished line together with any yet unprocessed data from the state.

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as first and an accumulator as second arguments.

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as argument.

Creates a CSV decoder state, prepopulated with the given RawData and using the given Options.

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as argument.

Decodes and returns the next CSV line in the given decoder state, together with an updated state which can be used for further incremental decoding.

Returns the default decode options.

Returns the default encode options.

Encodes the given CSV data structure into a CSV binary, using the given Options.

Creates a data provider from the given binary Bin to supply data in chunks of the given ChunkSize.

Creates a data provider from an open file given in IoDevice to supply data in chunks of the given ChunkSize.

Types

csv_field/0

-type csv_field() :: binary().

csv_line/0

-type csv_line() :: [csv_field()].

data/0

-type data() :: binary().

decode_options/0

-type decode_options() ::
          #{separator := byte(), enclosure := undefined | byte(), quote := undefined | byte()}.

encode_options/0

-type encode_options() ::
          #{separator := byte(),
            enclosure := undefined | byte(),
            quote := undefined | byte(),
            enclose := never | always | optional,
            end_of_line := binary()}.

provider/0

-type provider() :: fun(() -> end_of_data | {data(), provider()}).

state/0

-opaque state()

Functions

decode(RawData)

-spec decode(RawData :: data()) -> Lines :: [csv_line()].

Equivalent to decode(RawData, default_decode_options()).

decode(RawData, Options)

-spec decode(RawData :: data(), Options :: decode_options()) -> Lines :: [csv_line()].

Decodes the raw CSV document given in RawData, using the given Options, into a CSV data structure.

The return value is a list of CSV lines, which in turn are lists of CSV fields, which in turn are binaries representing the CSV field values.

decode_add_data(State0, RawData)

-spec decode_add_data(State0 :: state(), RawData :: data()) -> State1 :: state().

Adds another chunk of unprocessed RawData to the given decoder State.

Returns an updated state with the given data added.

decode_filter(Provider, Fun)

-spec decode_filter(Provider, Fun) -> Lines
                       when
                           Provider :: provider(),
                           Fun :: fun((Line) -> boolean()),
                           Line :: csv_line(),
                           Lines :: [csv_line()].

Equivalent to decode_filter(Provider, default_decode_options(), Fun).

decode_filter(Provider, Options, Fun)

-spec decode_filter(Provider, Options, Fun) -> Lines
                       when
                           Provider :: provider(),
                           Options :: decode_options(),
                           Fun :: fun((Line) -> boolean()),
                           Line :: csv_line(),
                           Lines :: [csv_line()].

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as argument.

Returns a list of CSV lines for which Fun returned true.

decode_filtermap(Provider, Fun)

-spec decode_filtermap(Provider, Fun) -> Result
                          when
                              Provider :: provider(),
                              Fun :: fun((Line) -> boolean() | {true, Mapped}),
                              Line :: csv_line(),
                              Mapped :: term(),
                              Result :: [Line | Mapped].

Equivalent to decode_filtermap(Provider, default_decode_options(), Fun).

decode_filtermap(Provider, Options, Fun)

-spec decode_filtermap(Provider, Options, Fun) -> Result
                          when
                              Provider :: provider(),
                              Options :: decode_options(),
                              Fun :: fun((Line) -> boolean() | {true, Mapped}),
                              Line :: csv_line(),
                              Mapped :: term(),
                              Result :: [Line | Mapped].

Combines the functionality of decode_filter and decode_map.

decode_flush(State)

-spec decode_flush(State :: state()) -> {Line :: undefined | csv_line(), Rest :: data()}.

Flushes a possibly unfinished line together with any yet unprocessed data from the state.

If there is no possibly unfinished line in the state, the atom undefined is returned instead of a line.

decode_fold(Provider, Fun, Acc0)

-spec decode_fold(Provider, Fun, Acc0) -> Acc1
                     when
                         Provider :: provider(),
                         Fun :: fun((Line, AccIn) -> AccOut),
                         Line :: csv_line(),
                         Acc0 :: term(),
                         Acc1 :: term(),
                         AccIn :: term(),
                         AccOut :: term().

Equivalent to decode_fold(Provider, default_decode_options(), Fun, Acc0).

decode_fold(Provider, Options, Fun, Acc0)

-spec decode_fold(Provider, Options, Fun, Acc0) -> Acc1
                     when
                         Provider :: provider(),
                         Options :: decode_options(),
                         Fun :: fun((Line, AccIn) -> AccOut),
                         Line :: csv_line(),
                         Acc0 :: term(),
                         Acc1 :: term(),
                         AccIn :: term(),
                         AccOut :: term().

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as first and an accumulator as second arguments.

Returns the final accumulator.

decode_foreach(Provider, Fun)

-spec decode_foreach(Provider, Fun) -> ok
                        when Provider :: provider(), Fun :: fun((Line) -> _), Line :: csv_line().

Equivalent to decode_foreach(Provider, default_decode_options(), Fun).

decode_foreach(Provider, Options, Fun)

-spec decode_foreach(Provider, Options, Fun) -> ok
                        when
                            Provider :: provider(),
                            Options :: decode_options(),
                            Fun :: fun((Line) -> _),
                            Line :: csv_line().

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as argument.

decode_init()

-spec decode_init() -> State :: state().

Equivalent to decode_init(<<>>, default_decode_options()).

decode_init(DataOrOptions)

-spec decode_init(DataOrOptions :: data() | decode_options()) -> State :: state().

Equivalent to decode_init(Data, default_decode_options()) or decode_init(<<>>, Options), respectively.

decode_init(RawData, Options)

-spec decode_init(RawData :: data(), Options :: decode_options()) -> State :: state().

Creates a CSV decoder state, prepopulated with the given RawData and using the given Options.

The return value can be used in the functions decode_add_data/2, decode_next_line/1 and decode_flush/1 to incrementally decode a CSV document.

See also: decode_add_data/2, decode_flush/1, decode_next_line/1.

decode_map(Provider, Fun)

-spec decode_map(Provider, Fun) -> Result
                    when
                        Provider :: provider(),
                        Fun :: fun((Line) -> Mapped),
                        Line :: csv_line(),
                        Mapped :: term(),
                        Result :: [Mapped].

Equivalent to decode_map(Provider, default_decode_options(), Fun).

decode_map(Provider, Options, Fun)

-spec decode_map(Provider, Options, Fun) -> Result
                    when
                        Provider :: provider(),
                        Options :: decode_options(),
                        Fun :: fun((Line) -> Mapped),
                        Line :: csv_line(),
                        Mapped :: term(),
                        Result :: [Mapped].

Successively calls the given function Fun with each CSV line decoded from the raw binary data provided by the given Provider as argument.

Returns a list of the values returned by Fun.

decode_next_line(State0)

-spec decode_next_line(State0 :: state()) -> {Result :: end_of_data | csv_line(), State1 :: state()}.

Decodes and returns the next CSV line in the given decoder state, together with an updated state which can be used for further incremental decoding.

If the decoder state is exhausted, the atom end_of_data is returned instead of a line. In this case, the function decode_add_data/2 can be used to add more data to the state, or decode_flush/1 can be used to flush a possibly unfinished line together with any yet unprocessed data from the state.

See also: decode_add_data/2, decode_flush/1.

default_decode_options()

-spec default_decode_options() -> decode_options().

Returns the default decode options.

  • separator: $,
  • enclosure: $"
  • quote: $"

default_encode_options()

-spec default_encode_options() -> encode_options().

Returns the default encode options.

  • separator: $,
  • enclosure: $"
  • quote: $"
  • enclose: optional
  • end_of_line: <<"\r\n">>

encode(Lines)

-spec encode(Lines :: [csv_line()]) -> RawData :: data().

Equivalent to encode(Lines, default_encode_options()).

encode(Lines, Options)

-spec encode(Lines :: [csv_line()], Options :: encode_options()) -> RawData :: data().

Encodes the given CSV data structure into a CSV binary, using the given Options.

get_binary_provider(Bin)

-spec get_binary_provider(Bin :: binary()) -> provider().

Equivalent to get_binary_provider(Bin, 1024).

get_binary_provider(Bin, ChunkSize)

-spec get_binary_provider(Bin :: binary(), ChunkSize :: pos_integer()) -> provider().

Creates a data provider from the given binary Bin to supply data in chunks of the given ChunkSize.

This provider can be used in decode_fold/3,4, decode_foreach/2,3, decode_filter/2,3, decode_map/2,3 and decode_filtermap/2,3.

get_file_provider(Filename)

-spec get_file_provider(Filename :: file:name_all()) -> provider().

Equivalent to get_file_provider(Filename, 1024).

get_file_provider(IoDevice, ChunkSize)

-spec get_file_provider(IoDevice :: file:io_device() | io:device(), ChunkSize :: pos_integer()) ->
                           provider().

Creates a data provider from an open file given in IoDevice to supply data in chunks of the given ChunkSize.

The file must have been opened with modes read and binary.

When the provider is exhausted, the position of the IoDevice is at eof.

The IoDevice is not closed implicitly by the provider, instead the code using this provider is responsible for closing it.

This provider can be used in decode_fold/3,4, decode_foreach/2,3, decode_filter/2,3, decode_map/2,3 and decode_filtermap/2,3.