Module estore

Event store time sharded over mitleople efiles.

Copyright © (C) 2016, Project-FiFo UG

Authors: Heinz Nikolaus Gies (heinz@project-fifo.net).

Description

Event store time sharded over mitleople efiles.

Data Types

estore()

abstract datatype: estore()

new_opt()

new_opt() = no_index | {file_size, tagged_time()} | {grace, tagged_time()}

tagged_time()

tagged_time() = pos_integer() | {pos_integer(), time_unit()}

time_unit()

time_unit() = ns | us | ms | s | m | h | d | w

Function Index

append/2Inserts one or more events into a event store.
close/1Closes all the files in an event store.
count/1Returns an APPROXIMATE count of elements in the estore.
delete/1Deletes the store and all files within it.
delete/2Delete all efiles that start before a certain point in time.
eid/0Utility function that generates a event id.
eid/1Creates a event id, using a sha1 hash on the node, the pid, erlang:unique_integer and a passed term.
event/1Creates a event tuple with timestamp, an eid and the evnt passed.
fold/3Folds the entire store, passing each event into the fold function.
fold/5Folds over a time range, passing each event into the fold function.
make_splits/3Splits a start end end based into chunks based on the file sized.
new/2Creates a new estore, the efiles will be stored in the given directory.
open/1
open/2
read/3Reads a time range from the estore, it will traverse multiple efiles if required.

Function Details

append/2

append(Es::[efile:event()], EStore::estore()) -> {ok, estore()}

Inserts one or more events into a event store. The events are expected to be ordered! Sharding will be done automatically and does not need to be handled outside of the estore.

As in fold strict order is not guarantted.

close/1

close(EStore::estore()) -> ok

Closes all the files in an event store.

count/1

count(EStore::estore()) -> {ok, non_neg_integer(), estore()}

Returns an APPROXIMATE count of elements in the estore. This uses the index as an indicator and by that should return a ballpark of how many entries exist.

delete/1

delete(EStore::estore()) -> ok

Deletes the store and all files within it.

delete/2

delete(Before::non_neg_integer(), EStore::estore()) -> {ok, estore()}

Delete all efiles that start before a certain point in time. this means some events that existed before 'Before' can still be in there.

eid/0

eid() -> <<_:32>>

Utility function that generates a event id. This is the same as eid/1 with the argument estore.

eid/1

eid(E::term()) -> <<_:32>>

Creates a event id, using a sha1 hash on the node, the pid, erlang:unique_integer and a passed term.

event/1

event(E) -> any()

Creates a event tuple with timestamp, an eid and the evnt passed. The timestamp does NOT have to be provided this function calls erlang:system_time/1 to get it.

fold/3

fold(Fun::efile:fold_fun(), Acc::any(), EStore::estore()) -> {ok, any(), estore()}

Folds the entire store, passing each event into the fold function. Strict order is not guarantted. Events might come out of order but will remain in the grace period.

fold/5

fold(Start::pos_integer(), End::pos_integer(), Fun::efile:fold_fun(), Acc::any(), EStore::estore()) -> {ok, any(), estore()}

Folds over a time range, passing each event into the fold function. Strict order is not guarantted. Events might come out of order but will remain in the grace period.

make_splits/3

make_splits(Time, End, Size) -> any()

Splits a start end end based into chunks based on the file sized. This is a utility function that can be used for distributing estore files.

new/2

new(Dir::string() | binary(), Opts::[new_opt()]) -> {ok, estore()} | {error, file_size | bad_index}

Creates a new estore, the efiles will be stored in the given directory.

open/1

open(Dir::string() | binary()) -> {ok, estore()} | {error, file_size | bad_index | enoent}

open/2

open(Dir::string() | binary(), Opts::[new_opt()]) -> {ok, estore()} | {error, file_size | bad_index | enoent}

read/3

read(Start::pos_integer(), End::pos_integer(), Estore::estore()) -> {ok, [efile:event()], estore()}

Reads a time range from the estore, it will traverse multiple efiles if required. Keep in mind that this will create one big aggregator for all events using fold/5 might be a better choice for larger datasets.

As in fold strict order is not guarantted.


Generated by EDoc, Sep 19 2016, 14:35:03.