Pockets (Pockets v1.6.0)
View SourcePockets
is a wrapper around Erlang :ets
and :dets
, built-in options for memory- and disk-based term storage.
It offers simple key/value storage using an interface similar to the Map
or Keyword
modules. This can be a useful
persistent cache for many use cases.
Note that this package and the libraries that underpin it may have limitations or specific behaviors that may affect
its suitability for various use-cases. For example, the limited support for concurrency provided by the :ets(3)
module is not yet provided by :dets
.
See also
Summary
Functions
Deletes the entry in table for a specific key
. Because this returns the table alias, you can
pipe multiple operations together.
Destroys the given table.
For disk-based (:dets
) tables, this will delete the backing file.
For memory-based (:ets
) tables, this destroys the table and its contents.
Checks if the given table is empty.
This will return true
if the given table does not exist.
Checks if the table exists in the Pockets
registry.
Akin to Enum.filter/2
, this function will pare down the contents of the given
table preserving only entries for which fun
returns a truthy value.
Similar to using Enum.filter/2
on maps, the fun
receives a tuple representing
the key and the value stored at that location.
Gets the value for a specific key
in the table.
Checks if the table has the given key
.
Increments the value at the given key
by the given step
(default 1).
If the table does not exist or the value at the given key
is not a number,
no action is taken.
Gets info about the given table, returned as either a %Pockets.EtsInfo{}
or a %Pockets.DetsInfo{}
struct.
An :error
tuple is returned if the table does not exist.
Gets info about the given item
in the table. The available items depend on the type of table.
An error tuple is returned if the table does not exist.
Gets a list of keys in the given table. For larger tables, consider using keys_stream/1
.
An error tuple is returned if the table does not exist.
Gets a list of keys in the given table as a stream. An error tuple is returned if the table does not exist.
This is a powerful function that lets you merge input
into an open table.
All data in the input will be added to the table: the keys in the input
have precedence
over pre-existing keys in the table.
Creates a new table either in memory (default) or on disk.
Open a table for use. The behavior and available options of this function vary depending on the storage mechanism of the table (i.e.memory or disk).
Puts the given value
under key
in given table.
Akin to Enum.reject/2
, this function will pare down the contents of the given
table deleting entries for which fun
returns a truthy value.
Similar to using Enum.filter/2
on maps, the fun
receives a tuple representing
the key and the value stored at that location.
Both :ets
and :dets
files can be saved to disk. You can use this function to persist an in-memory :ets
file to disk for later use, or you can use it to make a copy of an existing :dets
table.
Show all registered Pockets
tables.
Returns the size of the given table, measured by the number of entries. Zero is returned if the table does not exist.
Outputs the contents of the given table to a list.
If the table does not exist, an empty list is returned.
Although this is useful for debugging purposes, for larger data sets consider using to_stream/1
instead.
Outputs the contents of the table to a map.
If the table does not exist, an empty map is returned.
Although this is useful for debugging purposes, for larger data sets consider using to_stream/1
instead.
Outputs the contents of the table to a stream for lazy evaluation. Returns an error tuple if the table does not exist.
Truncates the given table; this removes all entries from the table while leaving its options intact.
Types
Functions
Closes the given table (i.e. the opposite of open/3
).
Only processes that have opened a table are allowed to close it.
For :ets
in-memory tables, this is the same as destroy/1
.
For :dets
disk-based tables, this will save any outstanding changes to the file.
Deletes the entry in table for a specific key
. Because this returns the table alias, you can
pipe multiple operations together.
Examples
iex> Pockets.new(:my_cache)
{:ok, :my_cache}
iex> Pockets.merge(:my_cache, %{a: "apple", b: "boy", c: "cat"})
:my_cache
iex> Pockets.to_map(:my_cache)
%{a: "apple", b: "boy", c: "cat"}
iex> Pockets.delete(:my_cache, :b)
:my_cache
iex> Pockets.to_map(:my_cache)
%{a: "apple", c: "cat"}
Destroys the given table.
For disk-based (:dets
) tables, this will delete the backing file.
For memory-based (:ets
) tables, this destroys the table and its contents.
Examples
iex> Pockets.new(:my_cache, "/tmp/cache.dets")
{:ok, :my_cache}
iex> Pockets.destroy(:my_cache)
:ok
Checks if the given table is empty.
This will return true
if the given table does not exist.
Examples
iex> {:ok, tid} = Pockets.new(:my_cache)
{:ok, :my_cache}
iex> Pockets.empty?(tid)
true
Checks if the table exists in the Pockets
registry.
Note: this function does not check whether an ETS or DETS table was
created outside of Pockets
(e.g. via ETS or DETS directly).
For those cases, see :ets.whereis/1
or :dets.info/1
.
@spec filter(table_alias :: alias(), (any() -> as_boolean(term()))) :: {:ok, non_neg_integer()} | {:error, String.t()}
Akin to Enum.filter/2
, this function will pare down the contents of the given
table preserving only entries for which fun
returns a truthy value.
Similar to using Enum.filter/2
on maps, the fun
receives a tuple representing
the key and the value stored at that location.
The result of a successful operation is an :ok
tuple including the count of
rows removed.
This Updates the Table In Place!
This operation updates the table in place, so it has the potential to delete every item in the table. Use it carefully!
If a safer alternative is needed, use Pockets.to_stream/1
to manipulate values
and feed the results into a new table.
Examples
iex> Pockets.new(:ex)
iex> Pockets.merge(:ex, %{a: 12, b: 7, c: 22, d: 8})
iex> Pockets.filter(:ex, fn {_, v} -> v > 10 end)
{:ok, 2}
iex> Pockets.to_map(:ex)
%{a: 12, c: 22}
See Also
Gets the value for a specific key
in the table.
For tables of type :set
, this function behaves like Map.get/3
: the value is returned (if present),
otherwise the default
is returned.
However, for tables of type :bag
and :duplicate_bag
, this function will always return a list and the
default
is ignored. This has to do with keyword lists being the underlying data structure powering all
of this.
The default value will be returned if the table does not exist.
Examples
iex> Pockets.new(:t1, :memory, type: :set)
{:ok, :t1}
iex> Pockets.get(:t1, :x, "Default Value")
"Default Value"
iex> Pockets.new(:t2, :memory, type: :bag)
{:ok, :t2}
iex> Pockets.get(:t2, :x, "Ignored Default Value")
[]
# Bag
iex> Pockets.new(:t_bag, :memory, type: :bag)
{:ok, :t_bag}
iex> Pockets.put(:t_bag, :c, "Cream")
:t_bag
iex> Pockets.put(:t_bag, :c, "Sugar")
:t_bag
iex> Pockets.put(:t_bag, :c, "Sugar")
:t_bag
iex> Pockets.get(:t_bag, :c)
["Cream", "Sugar"]
# Duplicate Bag
iex> Pockets.new(:t_dupe_bag, :memory, type: :duplicate_bag)
{:ok, :t_dupe_bag}
iex> Pockets.put(:t_dupe_bag, :c, "Cream")
:t_bag
iex> Pockets.put(:t_dupe_bag, :c, "Sugar")
:t_bag
iex> Pockets.put(:t_dupe_bag, :c, "Sugar")
:t_bag
iex> Pockets.get(:t_bag, :c)
["Cream", "Sugar", "Sugar"]
Checks if the table has the given key
.
false
is returned if the table does not exist.
@spec incr(alias(), key :: any(), step :: integer(), initial_value :: number()) :: alias() | {:error, String.t()}
Increments the value at the given key
by the given step
(default 1).
If the table does not exist or the value at the given key
is not a number,
no action is taken.
Examples
iex> Pockets.new(:inc_table)
:inc_table
|> Pockets.incr(:x)
|> Pockets.incr(:x)
|> Pockets.get(:x)
2
Decrement a value:
iex> Pockets.new(:decr_table)
:decr_table
|> Pockets.incr(:x, -1, 10)
|> Pockets.incr(:x, -1)
|> Pockets.get(:x)
8
@spec info(table_alias :: alias()) :: Pockets.EtsInfo.t() | Pockets.DetsInfo.t() | {:error, String.t()}
Gets info about the given table, returned as either a %Pockets.EtsInfo{}
or a %Pockets.DetsInfo{}
struct.
An :error
tuple is returned if the table does not exist.
Gets info about the given item
in the table. The available items depend on the type of table.
An error tuple is returned if the table does not exist.
Gets a list of keys in the given table. For larger tables, consider using keys_stream/1
.
An error tuple is returned if the table does not exist.
Gets a list of keys in the given table as a stream. An error tuple is returned if the table does not exist.
This is a powerful function that lets you merge input
into an open table.
All data in the input will be added to the table: the keys in the input
have precedence
over pre-existing keys in the table.
When the input
to be merged is...
- an alias for another table, the contents from that table are added to the given
table_alias
- a map, the contents from the map are added into the given
table_alias
- a keyword list, the contents from the list are added into the given
table_alias
Examples
# Merging a map into a table:
iex> Pockets.new(:my_cache)
{:ok, :my_cache}
iex> Pockets.merge(:my_cache, %{a: "apple", b: "boy", c: "cat"})
:my_cache
iex> Pockets.to_map(:my_cache)
%{a: "apple", b: "boy", c: "cat"}
# Merging a list into a table
iex> Pockets.new(:my_cache)
{:ok, :my_cache}
iex> Pockets.merge(:my_cache, [a: "apple", b: "boy", c: "cat"])
:my_cache
iex> Pockets.to_map(:my_cache)
%{a: "apple", b: "boy", c: "cat"}
# Merging two tables:
iex> Pockets.new(:my_first)
{:ok, :my_first}
iex> Pockets.merge(:my_first, %{a: "apple", b: "boy", c: "cat"})
:my_first
iex> Pockets.new(:my_second)
{:ok, :my_second}
iex> Pockets.merge(:my_second, %{x: "xray", y: "yellow", z: "zebra"})
:my_second
iex> Pockets.merge(:my_first, :my_second)
:my_first
iex> Pockets.to_map(:my_first)
%{a: "apple", b: "boy", c: "cat", x: "xray", y: "yellow", z: "zebra"}
@spec new(table_alias :: alias(), :memory | (file :: String.t()), opts :: keyword()) :: {:ok, alias()} | {:error, any()}
Creates a new table either in memory (default) or on disk.
The second argument specifies the storage mechanism for the table, either a path to a file (as a string)
for disk-backed tables (:dets
), or in :memory
for memory-backed tables (:ets
).
The available opts
are specific to the storage engine being used (i.e. to ets
or dets
).
In Memory
When creating a new memory-based table (i.e. the second argument is :memory
or is omitted),
the following options are supported:
:type
One of[:bag, :duplicate_bag, :set]
Default:set
:access
One of:public
|:protected
|:private
. Default::public
:named_table
boolean affects how the table is referenced. Iffalse
, the table alias will be a reference (not an atom). Default:true
:keypos
integer indicating the position of the element of each object to be used as key. Default:1
:read_concurrency
boolean. Default:true
:write_concurrency
boolean. Default:true
:decentralized_counters
boolean. Default:false
:compressed
boolean. Default:false
The full list of default options for memory-based tables are:
[type: :set, access: :public, named_table: true, keypos: 1, read_concurrency: true, write_concurrency: true, decentralized_counters: false, compressed: false]
The structure of the options is changed from the original to make them more compatible with Elixir conventions. See the original documentation for specifics about each option.
File-based Tables
When creating a new disk-based table (i.e. the second argument is a string path to a file), the following options are supported:
:type
One of[:bag, :duplicate_bag, :set]
Default:set
:access
One of:read
|:read_write
. Default::read_write
:auto_save
integer or:infinity
specifying the autosave interval (where:infinity
disables the feature). Default:180000
(3 minutes):keypos
integer indicating the position of the element of each object to be used as key. Default:1
:file
The name of the file to be opened. Defaults to the table name. (Use of this confusing option is not recommended):max_no_slots
The maximum number of slots to be used. Default:32 M
(million?):min_no_slots
Application performance can be enhanced with this flag by specifying the estimated number of different keys to be stored in the table. Default:256
(minimum):ram_file
boolean. Whether the table is to be kept in RAM. Keeping the table in RAM can sound like an anomaly, but it can enhance the performance of applications that open a table, insert a set of objects, and then close the table. When the table is closed, its contents are written to the disk file. Default:false
:repair
boolean or:force
. The flag specifies if the:dets
server invokes the automatic file reparation algorithm. Default:true
The default options for disk-based tables are:
[type: :set]
Examples
iex> Pockets.new(:ram_cache, :memory)
{:ok, :ram_cache}
iex> Pockets.new(:disk_cache, "/tmp/my.dets")
{:ok, :disk_cache}
Open a table for use. The behavior and available options of this function vary depending on the storage mechanism of the table (i.e.memory or disk).
Memory-based Tables
For memory-based tables open/3
is synonymous with new/3
.
Disk-based Tables
Because a file is involved, the :create?
option provides a bit more control over how to handle cases when the
file does not exist.
Options
:create?
Indicates whether a new table should be created if the one being requested does not already exist. Default:false
If :create?
is true
and the filepath indicated by the second argument does not exist,
open/3
is synonymous with new/3
.
Examples
iex> Pockets.open(:my_cache)
{:ok, :my_cache}
iex> Pockets.open(:my_cache, :memory)
{:ok, :my_cache}
iex> Pockets.open(:disk_cache, "/tmp/cache.dets")
{:ok, :disk_cache}
iex> Pockets.open(:boo, "/tmp/does_not_exist_yet.dets", create?: true)
{:ok, :boo}
Puts the given value
under key
in given table.
For tables of type :set
, this will behave like Map.put/3
.
Other table types aren't as straightforward. See the examples under get/3
Examples
# Set
iex> Pockets.new(:t_set)
{:ok, :t_set}
iex> Pockets.put(:t_set, :z, "Zulu")
:t_set
iex> Pockets.get(:t_set, :z)
"Zulu"
@spec reject(table_alias :: alias(), (any() -> as_boolean(term()))) :: {:ok, non_neg_integer()} | {:error, String.t()}
Akin to Enum.reject/2
, this function will pare down the contents of the given
table deleting entries for which fun
returns a truthy value.
Similar to using Enum.filter/2
on maps, the fun
receives a tuple representing
the key and the value stored at that location.
The result of a successful operation is an :ok
tuple including the count of
rows removed.
This Updates the Table In Place!
This operation updates the table in place, so it has the potential to delete every item in the table. Use it carefully!
If a safer alternative is needed, use Pockets.to_stream/1
to manipulate values
and feed the results into a new table.
Examples
iex> Pockets.new(:ex)
iex> Pockets.merge(:ex, %{a: 12, b: 7, c: 22, d: 8})
iex> Pockets.reject(:ex, fn {_, v} -> v > 10 end)
{:ok, 2}
iex> Pockets.to_map(:ex)
%{b: 7, d: 8}
See Also
Both :ets
and :dets
files can be saved to disk. You can use this function to persist an in-memory :ets
file to disk for later use, or you can use it to make a copy of an existing :dets
table.
The target file must not be in use by another table; if the target file exists this will return an error
unless the :overwrite?
option is set to true.
Options:
- overwrite? default:
false
Show all registered Pockets
tables.
Examples
iex> Pockets.show_tables()
[%Pockets.Table{alias: :my_cache, library: :ets, tid: :my_cache, type: :set}]
Returns the size of the given table, measured by the number of entries. Zero is returned if the table does not exist.
Outputs the contents of the given table to a list.
If the table does not exist, an empty list is returned.
Although this is useful for debugging purposes, for larger data sets consider using to_stream/1
instead.
Outputs the contents of the table to a map.
If the table does not exist, an empty map is returned.
Although this is useful for debugging purposes, for larger data sets consider using to_stream/1
instead.
Outputs the contents of the table to a stream for lazy evaluation. Returns an error tuple if the table does not exist.
Truncates the given table; this removes all entries from the table while leaving its options intact.
Examples
iex> Pockets.put(:my_cache, :a, "Apple") |> Pockets.put(:b, "boy") |> Pockets.put(:c, "Charlie")
:my_cache
iex> Pockets.truncate(:my_cache)
:my_cache
iex> Pockets.to_map(:my_cache)
%{}