A block signature of a basis binary, and its librsync wire encoding.
The basis is split into block_len-byte blocks (the last one may be
shorter). Each block contributes a {weak, strong} pair: its rolling
checksum and the first strong_sum_len bytes of its strong hash.
librsync defines four signature types, all supported:
weak | strong | magic | librsync name | strong_sum_len |
|---|---|---|---|---|
:rabinkarp | :blake2 | 0x72730147 | RS_RK_BLAKE2_SIG_MAGIC | 1..32 |
:rabinkarp | :md4 | 0x72730146 | RS_RK_MD4_SIG_MAGIC | 1..16 |
:rollsum | :blake2 | 0x72730137 | RS_BLAKE2_SIG_MAGIC | 1..32 |
:rollsum | :md4 | 0x72730136 | RS_MD4_SIG_MAGIC | 1..16 |
RabinKarp with BLAKE2b is the default and the default of librsync 2.3 and later. The rollsum and MD4 types exist for peers using older librsync defaults; MD4 is not collision-resistant.
Wire format, all integers big-endian:
u32 magic
u32 block_len
u32 strong_sum_len
repeated: u32 weak, strong_sum_len bytes strongThe format does not record the basis length, so a decoded signature cannot tell whether its last block is short.
index maps each weak checksum to the strong hashes seen with it, and each
of those to a block number. It is not part of the wire format;
build_index/1 fills it and Rexd.delta/2 calls that when the index is
missing. Building and querying it take constant time per block however the
checksums are distributed, so a signature crafted to share one weak
checksum across many blocks costs no more to process than any other.
Summary
Types
Weak checksum and truncated strong hash of one block.
Reasons decode/1 can fail.
Weak checksum to strong hash to the lowest block number carrying both.
Strong hash algorithm.
Rolling checksum algorithm.
Functions
Fills index, unless already present.
Decodes a librsync signature of any of the four types.
Encodes the signature in librsync wire format.
The librsync magic number of the signature's type.
Types
@type block() :: {non_neg_integer(), binary()}
Weak checksum and truncated strong hash of one block.
@type decode_error() :: :truncated_header | :truncated | {:bad_magic, non_neg_integer()} | {:invalid_block_len, non_neg_integer()} | {:invalid_strong_sum_len, non_neg_integer()}
Reasons decode/1 can fail.
@type index() :: %{ required(non_neg_integer()) => %{required(binary()) => non_neg_integer()} }
Weak checksum to strong hash to the lowest block number carrying both.
@type strong() :: :blake2 | :md4
Strong hash algorithm.
@type t() :: %Rexd.Signature{ block_len: pos_integer(), blocks: [block()], index: index() | nil, strong: strong(), strong_sum_len: 1..32, weak: weak() }
@type weak() :: :rabinkarp | :rollsum
Rolling checksum algorithm.
Functions
Fills index, unless already present.
Blocks that share both weak and strong checksum are stored once, under the lowest block number.
@spec decode(binary()) :: {:ok, t()} | {:error, decode_error()}
Decodes a librsync signature of any of the four types.
Encodes the signature in librsync wire format.
@spec magic(t()) :: non_neg_integer()
The librsync magic number of the signature's type.
iex> Rexd.Signature.magic(Rexd.signature(""))
0x72730147