Rexd.Signature (Rexd v1.0.0)

Copy Markdown View Source

A block signature of a basis binary, and its librsync wire encoding.

The basis is split into block_len-byte blocks (the last one may be shorter). Each block contributes a {weak, strong} pair: its rolling checksum and the first strong_sum_len bytes of its strong hash.

librsync defines four signature types, all supported:

weakstrongmagiclibrsync namestrong_sum_len
:rabinkarp:blake20x72730147RS_RK_BLAKE2_SIG_MAGIC1..32
:rabinkarp:md40x72730146RS_RK_MD4_SIG_MAGIC1..16
:rollsum:blake20x72730137RS_BLAKE2_SIG_MAGIC1..32
:rollsum:md40x72730136RS_MD4_SIG_MAGIC1..16

RabinKarp with BLAKE2b is the default and the default of librsync 2.3 and later. The rollsum and MD4 types exist for peers using older librsync defaults; MD4 is not collision-resistant.

Wire format, all integers big-endian:

u32 magic
u32 block_len
u32 strong_sum_len
repeated: u32 weak, strong_sum_len bytes strong

The format does not record the basis length, so a decoded signature cannot tell whether its last block is short.

index maps each weak checksum to the strong hashes seen with it, and each of those to a block number. It is not part of the wire format; build_index/1 fills it and Rexd.delta/2 calls that when the index is missing. Building and querying it take constant time per block however the checksums are distributed, so a signature crafted to share one weak checksum across many blocks costs no more to process than any other.

Summary

Types

Weak checksum and truncated strong hash of one block.

Reasons decode/1 can fail.

Weak checksum to strong hash to the lowest block number carrying both.

Strong hash algorithm.

t()

Rolling checksum algorithm.

Functions

Fills index, unless already present.

Decodes a librsync signature of any of the four types.

Encodes the signature in librsync wire format.

The librsync magic number of the signature's type.

Types

block()

@type block() :: {non_neg_integer(), binary()}

Weak checksum and truncated strong hash of one block.

decode_error()

@type decode_error() ::
  :truncated_header
  | :truncated
  | {:bad_magic, non_neg_integer()}
  | {:invalid_block_len, non_neg_integer()}
  | {:invalid_strong_sum_len, non_neg_integer()}

Reasons decode/1 can fail.

index()

@type index() :: %{
  required(non_neg_integer()) => %{required(binary()) => non_neg_integer()}
}

Weak checksum to strong hash to the lowest block number carrying both.

strong()

@type strong() :: :blake2 | :md4

Strong hash algorithm.

t()

@type t() :: %Rexd.Signature{
  block_len: pos_integer(),
  blocks: [block()],
  index: index() | nil,
  strong: strong(),
  strong_sum_len: 1..32,
  weak: weak()
}

weak()

@type weak() :: :rabinkarp | :rollsum

Rolling checksum algorithm.

Functions

build_index(sig)

@spec build_index(t()) :: t()

Fills index, unless already present.

Blocks that share both weak and strong checksum are stored once, under the lowest block number.

decode(bin)

@spec decode(binary()) :: {:ok, t()} | {:error, decode_error()}

Decodes a librsync signature of any of the four types.

encode(sig)

@spec encode(t()) :: iodata()

Encodes the signature in librsync wire format.

magic(signature)

@spec magic(t()) :: non_neg_integer()

The librsync magic number of the signature's type.

iex> Rexd.Signature.magic(Rexd.signature(""))
0x72730147