KafkaEx.Compression

Handles compression/decompression of messages.

See https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Compression

To add new compression types:

  1. Add the appropriate dependency to mix.exs (don’t forget to add it to the application list).
  2. Add the appropriate attribute value and compression_type atom.
  3. Add a decompress function clause.
  4. Add a compress function clause.

Summary

This function should pattern match on the compression_type atom and return the compressed data as well as the corresponding attribute value

This function should pattern match on the attribute value and return the decompressed data

Types

attribute_t :: integer
compression_type_t :: :snappy

Functions

compress(atom, data)

Specs

compress(compression_type_t, binary) :: {binary, attribute_t}

This function should pattern match on the compression_type atom and return the compressed data as well as the corresponding attribute value.

decompress(int, data)

Specs

decompress(attribute_t, binary) :: binary

This function should pattern match on the attribute value and return the decompressed data.

snappy_decompress_chunk(arg, so_far)