Nebulex.Adapters.Redis (Nebulex.Adapters.Redis v3.0.0-rc.1)
View SourceNebulex adapter for Redis. This adapter is implemented using Redix
(a Redis driver for Elixir).
The adapter provides three setup alternatives:
Standalone - The adapter establishes a pool of connections with a single Redis node. The
:standalone
is the default mode.Redis Cluster - Redis Cluster is a built-in feature in Redis since version 3, and it may be the most convenient and recommendable way to set up Redis in a cluster and have a distributed cache storage out-of-box. This adapter provides the
:redis_cluster
mode to set up Redis Cluster from the client-side automatically and be able to use it transparently.Built-in client-side cluster - The
:client_side_cluster
mode provides a simple client-side cluster implementation based on sharding distribution model.
Standalone
A cache that uses Redis is defined as follows:
defmodule MyApp.RedisCache do
use Nebulex.Cache,
otp_app: :nebulex,
adapter: Nebulex.Adapters.Redis
end
The configuration for the cache must be in your application environment,
usually defined in your config/config.exs
:
config :my_app, MyApp.RedisCache,
conn_opts: [
host: "127.0.0.1",
port: 6379
]
Redis Cluster
A cache that uses Redis Cluster can be defined as follows:
defmodule MyApp.RedisClusterCache do
use Nebulex.Cache,
otp_app: :nebulex,
adapter: Nebulex.Adapters.Redis
end
As you may notices, nothing has changed, it is defined the same as the standalone mode. The change is in the configuration:
config :my_app, MyApp.RedisClusterCache,
mode: :redis_cluster,
redis_cluster: [
configuration_endpoints: [
endpoint1_conn_opts: [
host: "127.0.0.1",
port: 6379,
# Add the password if 'requirepass' is on
password: "password"
],
...
]
]
Client-side Cluster
Same as the previous modes, a cache is defined as:
defmodule MyApp.ClusteredCache do
use Nebulex.Cache,
otp_app: :nebulex,
adapter: Nebulex.Adapters.Redis
end
The config:
config :my_app, MyApp.ClusteredCache,
mode: :client_side_cluster,
client_side_cluster: [
nodes: [
node1: [
pool_size: 10,
conn_opts: [
host: "127.0.0.1",
port: 9001
]
],
node2: [
pool_size: 4,
conn_opts: [
url: "redis://127.0.0.1:9002"
]
],
node3: [
conn_opts: [
host: "127.0.0.1",
port: 9003
]
],
...
]
]
Redis Proxy Alternative
Consider using a proxy instead, since it may provide more and better features. See the "Redis Proxy" section below for more information.
Redis Proxy
Another option for "Redis Cluster" or the built-in "Client-side cluster" is using a proxy such as Envoy proxy or Twemproxy on top of Redis. In this case, the proxy does the distribution work, and from the adparter's side (Nebulex.Adapters.Redis), it would be only configuration. Instead of connect the adapter against the Redis nodes, we connect it against the proxy nodes, this means, in the config, we setup the pool with the host and port pointing to the proxy.
Configuration options
In addition to Nebulex.Cache
config options, the adapter supports the
following options:
:mode
- Redis configuration mode.:standalone
- A single Redis instance. See the "Standalone" section in the module documentation for more options.:redis_cluster
- Redis Cluster setup. See the "Redis Cluster" section in the module documentation for more options.:client_side_cluster
- See the "Client-side Cluster" section in the module documentation for more options.
The default value is
:standalone
.:pool_size
(pos_integer/0
) - The number of connections that will be started by the adapter (based on the:mode
). The default value isSystem.schedulers_online()
.:serializer
- Custom serializer module implementing theNebulex.Adapters.Redis.Serializer
behaviour.:serializer_opts
(keyword/0
) - Custom serializer options. The default value is[]
.:encode_key
(keyword/0
) - Options for encoding the key. The default value is[]
.:encode_value
(keyword/0
) - Options for encoding the value. The default value is[]
.:decode_key
(keyword/0
) - Options for decoding the key. The default value is[]
.:decode_value
(keyword/0
) - Options for decoding the value. The default value is[]
.
:conn_opts
(keyword/0
) - Redis client options. SeeRedix
docs for more information about the options. The default value is[host: "127.0.0.1", port: 6379]
.:redis_cluster
- Required only when:mode
is set to:redis_cluster
. A keyword list of options.See "Redis Cluster options" section below.
:client_side_cluster
- Required only when:mode
is set to:client_side_cluster
. A keyword list of options.See "Client-side Cluster options" section below.
Redis Cluster options
The available options are:
:configuration_endpoints
- Required. A keyword list of named endpoints where the key is an atom as an identifier and the value is another keyword list of options (same as:conn_opts
).See "Redis Cluster" for more information.
:override_master_host
(boolean/0
) - Defines whether the given master host should be overridden with the configuration endpoint or not. Defaults tofalse
.The adapter uses the host returned by the "CLUSTER SHARDS" (Redis >= 7) or "CLUSTER SLOTS" (Redis < 7) command. One may consider set it to
true
for tests when using Docker for example, or when Redis nodes are behind a load balancer that Redis doesn't know the endpoint of. See Redis docs for more information.The default value is
false
.:keyslot
(function of arity 2) - A function to compute the hash slot for a given key and range. The default value is&Nebulex.Adapters.Redis.Cluster.Keyslot.hash_slot/2
.
Client-side Cluster options
The available options are:
:nodes
- Required. A keyword list of named nodes where the key is an atom as an identifier and the value is another keyword list of options (same as:conn_opts
).See "Client-side Cluster" for more information.
Shared runtime options
Since the adapter runs on top of Redix
, all commands accept their options
(e.g.: :timeout
, and :telemetry_metadata
). See Redix
docs for more
information.
Redis Cluster runtime options
The following options are only for the :redis_cluster
mode and apply to all
commands:
:lock_retries
- When the config manager is running and setting up the hash slot map, all Redis commands get blocked until the cluster is properly configured and the hash slot map is ready to use. This option defines the max retry attempts to acquire the lock before executing the command. Defaults to:infinity
.
Query API
Since the queryable API is implemented by using KEYS
command,
keep in mind the following caveats:
- Only keys can be queried.
- Only strings and predefined queries are allowed as query values.
See "KEYS" command.
Examples
iex> MyApp.RedisCache.put_all(%{
...> "firstname" => "Albert",
...> "lastname" => "Einstein",
...> "age" => 76
...> })
:ok
# returns key/value pairs by default
iex> MyApp.RedisCache.get_all!("**name**") |> Map.new()
%{"firstname" => "Albert", "lastname" => "Einstein"}
iex> MyApp.RedisCache.get_all!("**name**", select: :key)
["firstname", "lastname"]
iex> MyApp.RedisCache.get_all!("a??", select: :key)
["age"]
iex> MyApp.RedisCache.get_all!(select: :key)
["age", "firstname", "lastname"]
iex> MyApp.RedisCache.stream!("**name**", select: :key) |> Enum.to_list()
["firstname", "lastname"]
Deleting/counting keys
iex> MyApp.RedisCache.delete_all!({:in, ["foo", "bar"]})
2
iex> MyApp.RedisCache.count_all!({:in, ["foo", "bar"]})
2
Transactions
Transactions
Transaction support is not currently available in this adapter, but it is planned for future releases.
Using the adapter as a Redis client
Since the Redis adapter works on top of Redix
and provides features like
connection pools, "Redis Cluster", etc., it may also work as a Redis client.
The Redis API is quite extensive, and there are many useful commands we may
want to run, leveraging the Redis adapter features. Therefore, the adapter
provides additional functions to do so.
fetch_conn(opts \\ [])
The function accepts the following options:
:name
(atom/0
) - The name of the cache (in case you are using dynamic caches), otherwise it is not required (defaults to the cache module name).:key
(term/0
) - The key is used to compute the node where the command will be executed. It is only required for:redis_cluster
and:client_side_cluster
modes.
Let's see some examples:
iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LPUSH", "mylist", "hello"])
1
iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LPUSH", "mylist", "world"])
2
iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LRANGE", "mylist", "0", "-1"])
["hello", "world"]
When working with :redis_cluster
or :client_side_cluster
modes the option
:key
is required:
iex> {:ok, conn} = MyCache.fetch_conn(key: "mylist")
iex> Redix.pipeline!([
...> ["LPUSH", "mylist", "hello"],
...> ["LPUSH", "mylist", "world"],
...> ["LRANGE", "mylist", "0", "-1"]
...> ])
[1, 2, ["hello", "world"]]
Since these functions run on top of Redix
, they also accept their options
(e.g.: :timeout
, and :telemetry_metadata
). See Redix
docs for more
information.
Encoding/decoding functions
The following functions are available to encode/decode Elixir terms. It is useful whenever you want to work with Elixir terms in addition to strings or other specific Redis data types.
encode_key(name \\ __MODULE__, key)
- Encodes an Elixir term into a string. The argumentname
is optional and should be used in case of dynamic caches (Defaults to the defined cache module).encode_value(name \\ __MODULE__, value)
- Same asencode_key
but it is specific for encoding values, in case the encoding for keys and values are different.decode_key(name \\ __MODULE__, key)
- Decodes binary into an Elixir term. The argumentname
is optional and should be used in case of dynamic caches (Defaults to the defined cache module).decode_value(name \\ __MODULE__, value)
- Same asdecode_key
but it is specific for decoding values, in case the decoding for keys and values are different.
Let's see some examples:
iex> conn = MyCache.fetch_conn!()
iex> key = MyCache.encode_key({:key, "key"})
iex> value = MyCache.encode_value({:value, "value"})
iex> Redix.command!(conn, ["SET", key, value], timeout: 5000)
"OK"
iex> Redix.command!(conn, ["GET", key]) |> MyCache.decode_value()
{:value, "value"}
Adapter-specific telemetry events for the :redis_cluster
mode
Aside from the recommended Telemetry events by Nebulex.Cache
, this adapter
exposes the following Telemetry events for the :redis_cluster
mode:
telemetry_prefix ++ [:redis_cluster, :setup, :start]
- This event is specific to the:redis_cluster
mode. Before the configuration manager calls Redis to set up the cluster shards, this event should be invoked.The
:measurements
map will include the following::system_time
- The current system time in native units from calling:System.system_time()
.
A Telemetry
:metadata
map including the following fields::adapter_meta
- The adapter metadata.:pid
- The configuration manager PID.
telemetry_prefix ++ [:redis_cluster, :setup, :stop]
- This event is specific to the:redis_cluster
mode. After the configuration manager set up the cluster shards, this event should be invoked.The
:measurements
map will include the following::duration
- The time spent configuring the cluster. The measurement is given in the:native
time unit. You can read more about it in the docs forSystem.convert_time_unit/3
.
A Telemetry
:metadata
map including the following fields::adapter_meta
- The adapter metadata.:pid
- The configuration manager PID.:status
- The cluster setup status. If the cluster was configured successfully, the status will be set to:ok
, otherwise, will be set to:error
.:reason
- The status reason. When the status is:ok
, the reason is:succeeded
, otherwise, it is the error reason.
telemetry_prefix ++ [:redis_cluster, :setup, :exception]
- This event is specific to the:redis_cluster
mode. When an exception is raised while configuring the cluster, this event should be invoked.The
:measurements
map will include the following::duration
- The time spent configuring the cluster. The measurement is given in the:native
time unit. You can read more about it in the docs forSystem.convert_time_unit/3
.
A Telemetry
:metadata
map including the following fields::adapter_meta
- The adapter metadata.:pid
- The configuration manager PID.:kind
- The type of the error::error
,:exit
, or:throw
.:reason
- The reason of the error.:stacktrace
- The stacktrace.