TermStream (term_stream v0.1.0)

A Library for streaming Erlang terms into and out of binary streams. This is especially useful when storing large numbers of Erlang terms to file for later retrieval. When dealing with large files containing lots of Erlang terms, having to load the entire list into memory can be problematice. Using TermStream allows you to stream the terms one at a time out of the file.

Example

# To write a stream of terms to a file:

file = File.stream!("my_file", [:write, :binary])

other_stream
|> TermStream.serialize()
|> Stream.into(file)
|> Stream.run()

File.close(file)

# The get a stream out of the same file later:

File.stream!("my_file", 1024, [:read, :binary])
|> TermStream.deserialize()
|> Stream.run()

Summary

Functions

Takes a stream of binary data representing erlang terms (as written by TermStream.serialize/1 and returns a stream of the original terms.

Takes a stream of erlang terms and returns a stream of binary data representing those terms that is suitable for writing to a file without newlines or other delimiters.

Functions

deserialize(stream)

Takes a stream of binary data representing erlang terms (as written by TermStream.serialize/1 and returns a stream of the original terms.

serialize(stream, opts \\ [])

Takes a stream of erlang terms and returns a stream of binary data representing those terms that is suitable for writing to a file without newlines or other delimiters.

This function takes an optional opts argument that is passed to :erlang.term_to_binary/2. This can be used to control the compression or other parameters of the serialization. See the documentation for :erlang.term_to_binary/2 for more information.