This is the main module, which contains all Shards/ETS API functions, BUT works locally.
Shards is compatible with ETS API, most of the functions preserves the same ETS semantics, with some exception which you will find on each function doc.
Shards gives a top level view of a single logical ETS table,
but inside, that logical table is split in multiple physical
ETS tables called shards, where Shards = [0 .. N-1]
,
and N
is the number of shards into which you want to split
the table.
The K/V pairs are distributed across these shards, therefore, some of the functions does not follows the same semantics as the original ones ETS.
A good example of that are the query-based functions, which returns multiple results, and in case ofordered_set
, with
a particular order. E.g.:
select/2
, select/3
, select/1
select_reverse/2
, select_reverse/3
, select_reverse/1
match/2
, match/3
, match/1
match_object/2
, match_object/3
, match_object/1
For those cases, the order what results are returned is not guaranteed to be the same as the original ETS functions.
Additionally to the ETS functions, shards_local
module allows
to pass an extra argument, the State
. When shards
is
called without the State
, it must fetch the state
first,
and it is recovered doing an extra call to an ETS control table
owned by shards_owner_sup
. If any microsecond matters, you can
skip it call by calling shards_local
directly and passing
the State
. E.g.:
% create a table tab_name = shards:new(tab_name, [{n_shards, 4}]). % you can get the state at any time by calling: State = shards_state:get(tab_name). % normal way shards:lookup(table, key1). % calling shards_local directly shards_local:lookup(table, key1, State).Pools of shards can be added/removed dynamically. For example, using
shards:new/2
you can add more pools, and shards:delete/1
to remove the pool you wish.
continuation() = {Tab::atom(), MatchSpec::ets:match_spec(), Limit::pos_integer(), Shard::non_neg_integer(), Continuation::ets:continuation()}
Defines the convention to ets:select/1,3
continuation:
Tab
: Table name.MatchSpec
: The ets:match_spec()
.Limit
: Results limit.Shard
: Shards number.Continuation
: The ets:continuation()
.filename() = string() | binary() | atom()
info_item() = compressed | fixed | heir | keypos | memory | name | named_table | node | owner | protection | safe_fixed | size | stats | type | write_concurrency | read_concurrency | shards
info_tuple() = {compressed, boolean()} | {heir, pid() | none} | {keypos, pos_integer()} | {memory, non_neg_integer()} | {name, atom()} | {named_table, boolean()} | {node, node()} | {owner, pid()} | {protection, ets:access()} | {size, non_neg_integer()} | {type, ets:type()} | {write_concurrency, boolean()} | {read_concurrency, boolean()} | {shards, [atom()]}
option() = ets:type() | ets:access() | named_table | {keypos, pos_integer()} | {heir, pid(), HeirData::term()} | {heir, none} | tweaks() | shards_opt()
Create table options – used by new/2
.
shards_opt() = {scope, l | g} | {n_shards, pos_integer()} | {pick_shard_fun, shards_state:pick_fun()} | {pick_node_fun, shards_state:pick_fun()} | {restart_strategy, one_for_one | one_for_all}
Shards extended options.
tabinfo_item() = {name, atom()} | {type, ets:type()} | {protection, ets:access()} | {named_table, boolean()} | {keypos, non_neg_integer()} | {size, non_neg_integer()} | {extended_info, [md5sum | object_count]} | {version, {Major::non_neg_integer(), Minor::non_neg_integer()}} | {shards, [atom()]}
tweaks() = {write_concurrency, boolean()} | {read_concurrency, boolean()} | compressed
ETS tweaks option
all/0 | Equivalent to ets:all(). |
delete/1 |
This operation behaves like ets:delete/1 . |
delete/2 | Equivalent to delete(Tab, Key, shards_state:new()). |
delete/3 |
This operation behaves like ets:delete/2 . |
delete_all_objects/1 | Equivalent to delete_all_objects(Tab, shards_state:new()). |
delete_all_objects/2 |
This operation behaves like ets:delete_all_objects/1 . |
delete_object/2 | Equivalent to delete_object(Tab, Object, shards_state:new()). |
delete_object/3 |
This operation behaves like ets:delete_object/2 . |
file2tab/1 | Equivalent to file2tab(Filename, []). |
file2tab/2 |
Similar to shards:file2tab/2 . |
first/1 | Equivalent to first(Tab, shards_state:new()). |
first/2 |
This operation behaves similar to ets:first/1 . |
foldl/3 | Equivalent to foldl(Function, Acc0, Tab, shards_state:new()). |
foldl/4 |
This operation behaves like ets:foldl/3 . |
foldr/3 | Equivalent to foldr(Function, Acc0, Tab, shards_state:new()). |
foldr/4 |
This operation behaves like ets:foldr/3 . |
give_away/3 | Equivalent to give_away(Tab, Pid, GiftData, shards_state:new()). |
give_away/4 |
Equivalent to ets:give_away/3 for each shard table. |
i/0 | Equivalent to ets:i(). |
info/1 | Equivalent to info(Tab, shards_state:new()). |
info/2 |
If 2nd argument is info_item() this function behaves like
ets:info/2 , but if it is the shards_state:state() ,
it behaves like ets:info/1 . |
info/3 |
This operation is analogous to ets:info/2 . |
info_shard/1 | Equivalent to ets:info(ShardTab). |
info_shard/2 | Equivalent to ets:info(ShardTab, Item). |
insert/2 | Equivalent to insert(Tab, ObjOrObjL, shards_state:new()). |
insert/3 |
This operation behaves similar to ets:insert/2 , with a big
difference, it is not atomic. |
insert_new/2 | Equivalent to insert_new(Tab, ObjOrObjL, shards_state:new()). |
insert_new/3 |
This operation behaves like ets:insert_new/2 BUT it is not atomic,
which means if it fails inserting some K/V pair, only that K/V
pair is affected, the rest may be successfully inserted. |
is_compiled_ms/1 | Equivalent to ets:is_compiled_ms(Term). |
last/1 | Equivalent to last(Tab, shards_state:new()). |
last/2 |
This operation behaves similar to ets:last/1 . |
lookup/2 | Equivalent to lookup(Tab, Key, shards_state:new()). |
lookup/3 |
This operation behaves like ets:lookup/2 . |
lookup_element/3 | Equivalent to lookup_element(Tab, Key, Pos, shards_state:new()). |
lookup_element/4 |
This operation behaves like ets:lookup_element/3 . |
match/1 |
This operation behaves similar to ets:match/1 . |
match/2 | |
match/3 |
If 3rd argument is pos_integer() this function behaves
like ets:match/3 , but if it is the shards_state:state() ,
it behaves like ets:match/2 . |
match/4 |
This operation behaves similar to ets:match/3 . |
match_delete/2 | Equivalent to match_delete(Tab, Pattern, shards_state:new()). |
match_delete/3 |
This operation behaves like ets:match_delete/2 . |
match_object/1 |
This operation behaves similar to ets:match_object/1 . |
match_object/2 | Equivalent to match_object(Tab, Pattern, shards_state:new()). |
match_object/3 |
If 3rd argument is pos_integer() this function behaves like
ets:match_object/3 , but if it is the shards_state:state() ,
it behaves like ets:match_object/2 . |
match_object/4 |
This operation behaves similar to ets:match_object/3 . |
match_spec_compile/1 | Equivalent to ets:match_spec_compile(MatchSpec). |
match_spec_run/2 | Equivalent to ets:match_spec_run(List, CompiledMatchSpec). |
member/2 | Equivalent to member(Tab, Key, shards_state:new()). |
member/3 |
This operation behaves like ets:member/2 . |
new/2 |
This operation is analogous to ets:new/2 , BUT it behaves totally
different. |
next/2 | Equivalent to next(Tab, Key1, shards_state:new()). |
next/3 |
This operation behaves similar to ets:next/2 . |
prev/2 | Equivalent to prev(Tab, Key1, shards_state:new()). |
prev/3 |
This operation behaves similar to ets:prev/2 . |
rename/2 | Equivalent to rename(Tab, Name, shards_state:new()). |
rename/3 |
Equivalent to ets:rename/2 . |
safe_fixtable/2 | Equivalent to safe_fixtable(Tab, Fix, shards_state:new()). |
safe_fixtable/3 |
Equivalent to ets:safe_fixtable/2 for each shard table. |
select/1 |
This operation behaves similar to ets:select/1 . |
select/2 | Equivalent to select(Tab, MatchSpec, shards_state:new()). |
select/3 |
If 3rd argument is pos_integer() this function behaves like
ets:select/3 , but if it is the shards_state:state() ,
it behaves like ets:select/2 . |
select/4 |
This operation behaves similar to ets:select/3 . |
select_count/2 | Equivalent to select_count(Tab, MatchSpec, shards_state:new()). |
select_count/3 |
This operation behaves like ets:select_count/2 . |
select_delete/2 | Equivalent to select_delete(Tab, MatchSpec, shards_state:new()). |
select_delete/3 |
This operation behaves like ets:select_delete/2 . |
select_reverse/1 |
This operation behaves similar to ets:select_reverse/1 . |
select_reverse/2 | Equivalent to select_reverse(Tab, MatchSpec, shards_state:new()). |
select_reverse/3 |
If 3rd argument is pos_integer() this function behaves like
ets:select_reverse/3 , but if it is the shards_state:state() ,
it behaves like ets:select_reverse/2 . |
select_reverse/4 |
This operation behaves similar to ets:select_reverse/3 . |
setopts/2 | Equivalent to setopts(Tab, Opts, shards_state:new()). |
setopts/3 |
Equivalent to ets:setopts/2 for each shard table. |
tab2file/2 | Equivalent to tab2file(Tab, Filenames, shards_state:new()). |
tab2file/3 | Equivalent to tab2file / 4. |
tab2file/4 |
Similar to ets:tab2file/3 , but it behaves different. |
tab2list/1 | Equivalent to tab2list(Tab, shards_state:new()). |
tab2list/2 |
This operation behaves like ets:tab2list/1 . |
tabfile_info/1 |
This operation is analogous to ets:tabfile_info/1 , but it
adds info about the shards {shards, [atom()]} . |
table/1 | Equivalent to table(Tab, shards_state:new()). |
table/2 | Equivalent to table / 3. |
table/3 |
Similar to ets:table/2 , but it returns a list of ets:table/2
responses, one for each shard table. |
take/2 | Equivalent to take(Tab, Key, shards_state:new()). |
take/3 |
This operation behaves like ets:take/2 . |
test_ms/2 | Equivalent to ets:test_ms(Tuple, MatchSpec). |
update_counter/3 | Equivalent to update_counter(Tab, Key, UpdateOp, shards_state:new()). |
update_counter/4 |
This operation behaves like ets:update_counter/3 . |
update_counter/5 |
This operation behaves like ets:update_counter/4 . |
update_element/3 | Equivalent to update_element(Tab, Key, ElementSpec, shards_state:new()). |
update_element/4 |
This operation behaves like ets:update_element/3 . |
all() -> any()
Equivalent to ets:all().
delete(Tab::atom()) -> true
This operation behaves like ets:delete/1
.
See also: ets:delete/1.
delete(Tab, Key) -> any()
Equivalent to delete(Tab, Key, shards_state:new()).
delete(Tab, Key, State) -> true
This operation behaves like ets:delete/2
.
See also: ets:delete/2.
delete_all_objects(Tab) -> any()
Equivalent to delete_all_objects(Tab, shards_state:new()).
delete_all_objects(Tab, State) -> true
This operation behaves like ets:delete_all_objects/1
.
See also: ets:delete_all_objects/1.
delete_object(Tab, Object) -> any()
Equivalent to delete_object(Tab, Object, shards_state:new()).
delete_object(Tab, Object, State) -> true
This operation behaves like ets:delete_object/2
.
See also: ets:delete_object/2.
file2tab(Filename) -> any()
Equivalent to file2tab(Filename, []).
file2tab(Filename, Options) -> Response
Similar to shards:file2tab/2
. Moreover, it restores the
supervision tree for the shards
corresponding to the given
file, such as if they had been created using shards:new/2,3
.
See also: ets:file2tab/2.
first(Tab) -> any()
Equivalent to first(Tab, shards_state:new()).
first(Tab, State) -> Key | '$end_of_table'
This operation behaves similar to ets:first/1
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:first/1.
foldl(Function, Acc0, Tab) -> any()
Equivalent to foldl(Function, Acc0, Tab, shards_state:new()).
foldl(Function, Acc0, Tab, State) -> Acc1
This operation behaves like ets:foldl/3
.
See also: ets:foldl/3.
foldr(Function, Acc0, Tab) -> any()
Equivalent to foldr(Function, Acc0, Tab, shards_state:new()).
foldr(Function, Acc0, Tab, State) -> Acc1
This operation behaves like ets:foldr/3
.
See also: ets:foldr/3.
give_away(Tab, Pid, GiftData) -> any()
Equivalent to give_away(Tab, Pid, GiftData, shards_state:new()).
give_away(Tab, Pid, GiftData, State) -> true
Equivalent to ets:give_away/3
for each shard table. It returns
a boolean()
instead that just true
. Returns true
if the
function was applied successfully on each shard, otherwise
false
is returned.
WARNING: It is not recommended execute
this function, since it might cause an unexpected behavior.
Once this function is executed, shards
doesn't control/manage
the ETS shards anymore. So from this point, you should use
ETS API instead. Also it is recommended to run shards:delete/1
after run this function.
See also: ets:give_away/3.
i() -> any()
Equivalent to ets:i().
info(Tab) -> any()
Equivalent to info(Tab, shards_state:new()).
info(Tab, StateOrItem) -> Result
If 2nd argument is info_item()
this function behaves like
ets:info/2
, but if it is the shards_state:state()
,
it behaves like ets:info/1
.
{shards, [atom()]}
.
See also: ets:info/1, ets:info/2.
info(Tab, Item, State) -> Value
This operation is analogous to ets:info/2
.
See also: ets:info/2.
info_shard(ShardTab) -> any()
Equivalent to ets:info(ShardTab).
info_shard(ShardTab, Item) -> any()
Equivalent to ets:info(ShardTab, Item).
insert(Tab, ObjOrObjL) -> any()
Equivalent to insert(Tab, ObjOrObjL, shards_state:new()).
insert(Tab, ObjOrObjL, State) -> true
This operation behaves similar to ets:insert/2
, with a big
difference, it is not atomic. This means if it fails
inserting some K/V pair, previous inserted KV pairs are not
rolled back.
See also: ets:insert/2.
insert_new(Tab, ObjOrObjL) -> any()
Equivalent to insert_new(Tab, ObjOrObjL, shards_state:new()).
insert_new(Tab, ObjOrObjL, State) -> Result
This operation behaves like ets:insert_new/2
BUT it is not atomic,
which means if it fails inserting some K/V pair, only that K/V
pair is affected, the rest may be successfully inserted.
ObjectOrObjects
is a list.
See also: ets:insert_new/2.
is_compiled_ms(Term) -> any()
Equivalent to ets:is_compiled_ms(Term).
last(Tab) -> any()
Equivalent to last(Tab, shards_state:new()).
last(Tab, State) -> Key | '$end_of_table'
This operation behaves similar to ets:last/1
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:last/1.
lookup(Tab, Key) -> any()
Equivalent to lookup(Tab, Key, shards_state:new()).
lookup(Tab, Key, State) -> Result
This operation behaves like ets:lookup/2
.
See also: ets:lookup/2.
lookup_element(Tab, Key, Pos) -> any()
Equivalent to lookup_element(Tab, Key, Pos, shards_state:new()).
lookup_element(Tab, Key, Pos, State) -> Elem
This operation behaves like ets:lookup_element/3
.
See also: ets:lookup_element/3.
match(Continuation) -> Response
This operation behaves similar to ets:match/1
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:match/1.
match(Tab, Pattern) -> any()
match(Tab, Pattern, StateOrLimit) -> Response
If 3rd argument is pos_integer()
this function behaves
like ets:match/3
, but if it is the shards_state:state()
,
it behaves like ets:match/2
.
See also: ets:match/2, ets:match/3.
match(Tab, Pattern, Limit, State) -> Response
This operation behaves similar to ets:match/3
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:match/3.
match_delete(Tab, Pattern) -> any()
Equivalent to match_delete(Tab, Pattern, shards_state:new()).
match_delete(Tab, Pattern, State) -> true
This operation behaves like ets:match_delete/2
.
See also: ets:match_delete/2.
match_object(Continuation) -> Response
This operation behaves similar to ets:match_object/1
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:match_object/1.
match_object(Tab, Pattern) -> any()
Equivalent to match_object(Tab, Pattern, shards_state:new()).
match_object(Tab, Pattern, StateOrLimit) -> Response
If 3rd argument is pos_integer()
this function behaves like
ets:match_object/3
, but if it is the shards_state:state()
,
it behaves like ets:match_object/2
.
See also: ets:match_object/2, ets:match_object/3.
match_object(Tab, Pattern, Limit, State) -> Response
This operation behaves similar to ets:match_object/3
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:match_object/3.
match_spec_compile(MatchSpec) -> any()
Equivalent to ets:match_spec_compile(MatchSpec).
match_spec_run(List, CompiledMatchSpec) -> any()
Equivalent to ets:match_spec_run(List, CompiledMatchSpec).
member(Tab, Key) -> any()
Equivalent to member(Tab, Key, shards_state:new()).
member(Tab, Key, State) -> boolean()
This operation behaves like ets:member/2
.
See also: ets:member/2.
new(Name, Options) -> Name
This operation is analogous to ets:new/2
, BUT it behaves totally
different. When this function is called, instead of create a single
table, a new supervision tree is created and added to shards_sup
.
This supervision tree has a main supervisor shards_sup
which
creates a control ETS table and also creates N
number of
shards_owner
(being N
the number of shards). Each shards_owner
creates an ETS table to represent each shard, so this gen_server
acts as the table owner.
Finally, when you create a table, internally N
physical tables
are created (one per shard), but shards
encapsulates all this
and you see only one logical table (similar to how a distributed
storage works).
NumShards = number of schedulers
.
See also: ets:new/2.
next(Tab, Key1) -> any()
Equivalent to next(Tab, Key1, shards_state:new()).
next(Tab, Key1, State) -> Key2 | '$end_of_table'
This operation behaves similar to ets:next/2
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning. It raises a bad_pick_fun_ret
exception in case of pick fun returns any
.
See also: ets:next/2.
prev(Tab, Key1) -> any()
Equivalent to prev(Tab, Key1, shards_state:new()).
prev(Tab, Key1, State) -> Key2 | '$end_of_table'
This operation behaves similar to ets:prev/2
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:prev/2.
rename(Tab, Name) -> any()
Equivalent to rename(Tab, Name, shards_state:new()).
rename(Tab, Name, State) -> Name | no_return()
Equivalent to ets:rename/2
.
See also: ets:rename/2.
safe_fixtable(Tab, Fix) -> any()
Equivalent to safe_fixtable(Tab, Fix, shards_state:new()).
safe_fixtable(Tab, Fix, State) -> boolean()
Equivalent to ets:safe_fixtable/2
for each shard table.
It returns a boolean()
instead that just true
.
Returns true
if the function was applied successfully
on each shard, otherwise false
is returned.
See also: ets:safe_fixtable/2.
select(Continuation) -> Response
This operation behaves similar to ets:select/1
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:select/1.
select(Tab, MatchSpec) -> any()
Equivalent to select(Tab, MatchSpec, shards_state:new()).
select(Tab, MatchSpec, StateOrLimit) -> Response
If 3rd argument is pos_integer()
this function behaves like
ets:select/3
, but if it is the shards_state:state()
,
it behaves like ets:select/2
.
See also: ets:select/2, ets:select/3.
select(Tab, MatchSpec, Limit, State) -> Response
This operation behaves similar to ets:select/3
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:select/3.
select_count(Tab, MatchSpec) -> any()
Equivalent to select_count(Tab, MatchSpec, shards_state:new()).
select_count(Tab, MatchSpec, State) -> NumMatched
This operation behaves like ets:select_count/2
.
See also: ets:select_count/2.
select_delete(Tab, MatchSpec) -> any()
Equivalent to select_delete(Tab, MatchSpec, shards_state:new()).
select_delete(Tab, MatchSpec, State) -> NumDeleted
This operation behaves like ets:select_delete/2
.
See also: ets:select_delete/2.
select_reverse(Continuation) -> Response
This operation behaves similar to ets:select_reverse/1
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:select_reverse/1.
select_reverse(Tab, MatchSpec) -> any()
Equivalent to select_reverse(Tab, MatchSpec, shards_state:new()).
select_reverse(Tab, MatchSpec, StateOrLimit) -> Response
If 3rd argument is pos_integer()
this function behaves like
ets:select_reverse/3
, but if it is the shards_state:state()
,
it behaves like ets:select_reverse/2
.
See also: ets:select_reverse/2, ets:select_reverse/3.
select_reverse(Tab, MatchSpec, Limit, State) -> Response
This operation behaves similar to ets:select_reverse/3
.
The order in which results are returned, might be not the same
as the original ETS function. Remember shards architecture
described at the beginning.
See also: ets:select_reverse/3.
setopts(Tab, Opts) -> any()
Equivalent to setopts(Tab, Opts, shards_state:new()).
setopts(Tab, Opts, State) -> boolean()
Equivalent to ets:setopts/2
for each shard table. It returns
a boolean()
instead that just true
. Returns true
if the
function was applied successfully on each shard, otherwise
false
is returned.
See also: ets:setopts/2.
tab2file(Tab, Filename) -> any()
Equivalent to tab2file(Tab, Filenames, shards_state:new()).
tab2file(Tab, Filename, Options) -> any()
Equivalent to tab2file / 4.
tab2file(Tab, Filename, Options, State) -> Response
Similar to ets:tab2file/3
, but it behaves different.
This function generates one file per shard using ets:tab2file/3
,
and also generate a master file with the given Filename
that
holds the information about the other shards files in order to
be able to recover them using ets:file2tab/1,2
.
See also: ets:tab2file/3.
tab2list(Tab) -> any()
Equivalent to tab2list(Tab, shards_state:new()).
tab2list(Tab, State) -> [Object]
This operation behaves like ets:tab2list/1
.
See also: ets:tab2list/1.
tabfile_info(Filename) -> Response
This operation is analogous to ets:tabfile_info/1
, but it
adds info about the shards {shards, [atom()]}
.
See also: ets:tabfile_info/1.
table(Tab) -> any()
Equivalent to table(Tab, shards_state:new()).
table(Tab, Options) -> any()
Equivalent to table / 3.
table(Tab, Options, State) -> [QueryHandle]
Similar to ets:table/2
, but it returns a list of ets:table/2
responses, one for each shard table.
See also: ets:table/2.
take(Tab, Key) -> any()
Equivalent to take(Tab, Key, shards_state:new()).
take(Tab, Key, State) -> [Object]
This operation behaves like ets:take/2
.
See also: ets:take/2.
test_ms(Tuple, MatchSpec) -> any()
Equivalent to ets:test_ms(Tuple, MatchSpec).
update_counter(Tab, Key, UpdateOp) -> any()
Equivalent to update_counter(Tab, Key, UpdateOp, shards_state:new()).
update_counter(Tab, Key, UpdateOp, State) -> Result
This operation behaves like ets:update_counter/3
.
See also: ets:update_counter/3.
update_counter(Tab, Key, UpdateOp, Default, State) -> Result
This operation behaves like ets:update_counter/4
.
See also: ets:update_counter/4.
update_element(Tab, Key, ElementSpec) -> any()
Equivalent to update_element(Tab, Key, ElementSpec, shards_state:new()).
update_element(Tab, Key, ElementSpec, State) -> boolean()
This operation behaves like ets:update_element/3
.
See also: ets:update_element/3.
Generated by EDoc, Jul 11 2017, 14:57:01.