clickhousex v0.5.0 Clickhousex.Codec.Binary.Extractor View Source
Allows modules that use
this module to create efficient extractor functions that speak clickhouse's binary protocol.
To define extractors, annotate a function with the extract
attribute like this:
@extract length: :varint
def extract_length(<<data::binary>>, length, other_param) do
do_something_with_length(data, length, other_param)
end
def do_something_with_length(_data, length, other_param) do
{other_param, length}
end
In the above example, a function named extract_length/2
will be created, which, when passed a binary, will
extract the length varint from it, and call the function above, passing the unparsed part of the binary and the extracted
length varint to it.
Usage looks like this
{:ok, binary_from_network} = :gen_tcp.recv(conn, 0)
{:this_is_passed_along, length} = extract_length(binary_from_network, :this_is_passed_along)
If there isn't enough data to parse, a resume tuple is returned. The second element of the tuple is a function that when called with more data, picks up the parse operation where it left off.
{:resume, resume_fn} = extract_length(<<>>, :this_is_passed_along)
{:ok, data} = :gen_tcp.recv(conn, 0)
{:this_is_passed_along, length} = resume_fn.(data)
Performance
All functions generated by this module take advantage of binary optimizations, resuse match contexts and won't create sub-binaries.
Completeness
The following extractors are implemented:
- Variable length integers
:varint
- Signed integers:
:i8
,:i16
,:i32
,i64
- Unsigned integers:
:u8
,:u16
,:u32
,:u64
- Floats:
:f32
,:f64
- Strings:
:string
- Booleans:
:boolean
- Dates:
:date
,:datetime
- Lists of the above scalar types
{:list, scalar}
- Nullable instances of all the above
{:nullable, scalar}
or{:list, {:nullable, scalar}}