Streaming signature, delta and patch.
Each function takes an enumerable of binaries, split at arbitrary
boundaries, and returns a lazy enumerable of binaries. Concatenated, the
output is the librsync wire format (signature/2, delta/2) or the
rebuilt data (patch/3). Nothing is read until the result is consumed.
Memory
signature/2holds less than one block of input.delta/2holds the signature, one block, the current input chunk (small chunks are grouped up to 64 KiB, larger ones are used as given), and the unmatched bytes not yet emitted. Those are emitted as a literal command once they reach 64 KiB at the end of an input chunk.patch/3holds a partial command header. Literal data is passed through as it arrives, and copies are read from the basis in pieces of at most 64 KiB.
Differences from the whole-binary functions
signature/2 produces exactly the bytes of Rexd.Signature.encode/1.
delta/2 finds the same matches as Rexd.delta/2, but may split a long
unmatched run into several literal commands, as librsync does; the rebuilt
output is identical.
Errors
Invalid options raise ArgumentError when the function is called. Invalid
input can only be detected while the stream is consumed, so it raises
Rexd.StreamError then, with the same reason terms as the tuple-returning
functions.
Summary
Types
Reads length bytes of the basis starting at offset.
Functions
Streams the encoded delta from the basis described by signature to the
new data read from enumerable.
Streams the data rebuilt by applying the encoded delta read from
enumerable to a basis.
Streams the encoded signature of the basis read from enumerable.
Types
@type basis_fun() :: (non_neg_integer(), pos_integer() -> binary())
Reads length bytes of the basis starting at offset.
Functions
@spec delta(Rexd.Signature.t(), Enumerable.t(), keyword()) :: Enumerable.t()
Streams the encoded delta from the basis described by signature to the
new data read from enumerable.
Options
:on_stats- a function called withRexd.Delta.Statsonce the last command has been produced, before the end marker is emitted.iex> basis = String.duplicate("0123456789", 50) iex> new = ["prefix ", basis, " suffix"] iex> sig = Rexd.signature(basis, block_len: 16) iex> {:ok, delta} = sig |> Rexd.Stream.delta(new) |> Enum.join() |> Rexd.Delta.decode() iex> Rexd.patch(basis, delta) == {:ok, IO.iodata_to_binary(new)} true
@spec patch(basis_fun(), Enumerable.t(), keyword()) :: Enumerable.t()
Streams the data rebuilt by applying the encoded delta read from
enumerable to a basis.
The basis is read through basis_fun, called as basis_fun.(offset, length)
with length of at most 64 KiB. It must return exactly length bytes, or a
shorter binary when the range extends past the end of the basis. For a file,
fn offset, length -> :file.pread(io_device, offset, length) end wrapped to
return the data (or <<>> on :eof) is enough.
Takes the options of Rexd.patch/3. Raises Rexd.StreamError while
consumed on an invalid delta, a copy past the end of the basis, or output
beyond :max_size.
iex> basis = "the quick brown fox"
iex> sig = Rexd.signature(basis, block_len: 4)
iex> encoded = sig |> Rexd.Stream.delta(["the quick red fox"]) |> Enum.join()
iex> read = fn offset, length -> binary_part(basis, offset, min(length, byte_size(basis) - offset)) end
iex> read |> Rexd.Stream.patch([encoded]) |> Enum.join()
"the quick red fox"@spec signature(Enumerable.t(), keyword()) :: Enumerable.t()
Streams the encoded signature of the basis read from enumerable.
Takes the options of Rexd.signature/2.
iex> basis = String.duplicate("abcdefgh", 100)
iex> streamed = ["abcdefgh" |> String.duplicate(40), String.duplicate("abcdefgh", 60)]
iex> streamed |> Rexd.Stream.signature(block_len: 64) |> Enum.join() ==
...> basis |> Rexd.signature(block_len: 64) |> Rexd.Signature.encode() |> IO.iodata_to_binary()
true