Copyright © 2019 Klarna Bank AB (publ)
This module defines the kflow_gen_buffer behaviour.
Required callback functions: in/4, out/2.
Optional callback functions: init/1, terminate/2.
This module implements a stateful buffer-like stream processing node.
One has to create a callback module with kflow_gen_buffer
behavior.
init
and terminate
callbacks are similar to those in kflow_gen_map
or kflow_gen_filter
behaviors.
in
callback is invoked for each incoming message from the
upstream. It takes 4 arguments:
init
callback and can be mutated in the callbacks.Return value should be a tuple {Flush, NextState}
where Flush
can be atoms keep
, flush
or reject
.
keep
means that the buffer should keep collecting upstream
messages without producing anything downstream.
flush
means that the buffer is ready to produce a message
downstream.
reject
means the last upstream message was incompatible with the
data that had been buffered so far. (E.g. schema of the data was
different). In this case previous state is flushed and the
last message is replayed from blank state.
out
callback is used to produce a message downstream. It is
invoked after in
callback returns flush
or reject
, or when
flush is implicitly requested by low-level control logic. It takes
two arguments: first one is current state of the callback module
and the second one is initial configuration.
It should output a tuple {DownstreamMessage, NextState}
-module(my_map). -behavior(kflow_gen_map). -export([init/1, map/3, terminate/1]). init(_Config) -> []. in(Offset, Message, State, Config) -> N = maps:get(buffer_size, Config), Flush = if length(State) >= N -> flush; true -> keep end, {Flush, [Message|State]}. out(State, _Config) -> Output = lists:reverse(State), NewState = [], {Output, NewState}. terminate(_State) -> ok.
Generated by EDoc