ExAws.Kinesis.Adapter behaviour

The purpose of this module is to surface the ExAws.Kinesis API with a single configuration chosen, such that it does not need passed in with every request.

Usage:

defmodule MyApp.Kinesis do
  use ExAws.Kinesis.Adapter, otp_app: :my_otp_app
end

In your config

config :my_otp_app, ExAws,
  kinesis:  [], # kinesis config goes here
  dynamodb: [], # you get the idea

You can now use MyApp.Kinesis as the root module for the Kinesis api without needing to pass in a particular configuration. This enables different otp apps to configure their AWS configuration separately.

The alignment with a particular OTP app however is entirely optional. The following also works:

defmodule MyApp.Kinesis do
  use ExAws.Kinesis.Adapter

  def config do
    [
      kinesis:  [], # kinesis config goes here
    ]
  end
end

This is in fact how the functions in ExAws.Kinesis that do not require a config work. Default config values can be found in ExAws.Config

http://docs.aws.amazon.com/kinesis/latest/APIReference/API_Operations.html

Callbacks

add_tags_to_stream/2

Specs:

  • add_tags_to_stream(name :: iodata, tags :: %{}) :: term

Add tags to stream

config/0

Specs:

By default the config is obtained by

Application.get_env(@otp_app, ExAws)[:kinesis]

via a function created when using ExAws.Kinesis.Adapter is called.

See ExAws.Kinesis.config/0 for an example that overrides this default

create_stream/1

Specs:

  • create_stream(stream_name :: iodata) :: term

Creates stream

create_stream/2

Specs:

  • create_stream(stream_name :: iodata, shard_count :: pos_integer) :: term
delete_stream/1

Specs:

  • delete_stream(stream_name :: iodata) :: term

Deletes stream

describe_stream/1

Specs:

  • describe_stream(stream_name :: iodata) :: term

Describe Stream

describe_stream/2

Specs:

  • describe_stream(stream_name :: iodata, opts :: %{}) :: term
get_records/1

Specs:

  • get_records(stream_name :: iodata) :: term

Get stream records

get_records/2

Specs:

  • get_records(stream_name :: iodata, opts :: %{}) :: term
get_shard_iterator/3

Specs:

  • get_shard_iterator(stream_name :: iodata, shard_id :: iodata, shard_iterator_type :: iodata) :: term

Get a shard iterator @type shard_iterator_types :: “AT_SEQUENCE_NUMBER” | “AFTER_SEQUENCE_NUMBER” | “TRIM_HORIZON” | “LATEST”

get_shard_iterator/4

Specs:

  • get_shard_iterator(stream_name :: iodata, shard_id :: iodata, shard_iterator_type :: iodata, opts :: %{}) :: term
list_streams/0

Specs:

  • list_streams :: term

Lists streams

list_tags_for_stream/1

Specs:

  • list_tags_for_stream(name :: iodata) :: term

Add tags to stream

list_tags_for_stream/2

Specs:

  • list_tags_for_stream(name :: iodata, opts :: %{}) :: term
merge_shards/3

Specs:

  • merge_shards(stream_name :: iodata, adjacent_shard_id :: iodata, shard_id :: iodata) :: term

Merge adjacent shards

put_record/3

Specs:

  • put_record(stream_name :: iodata, partition_key :: iodata, blob :: iodata) :: term

Puts a record on a stream

put_record/4

Specs:

  • put_record(stream_name :: iodata, partition_key :: iodata, blob :: iodata, opts :: %{}) :: term
put_records/2

Specs:

  • put_records(stream_name :: iodata, records :: [%{}]) :: term

Put multiple records on a stream

remove_tags_from_stream/2

Specs:

  • remove_tags_from_stream(name :: iodata, tag_keys :: [binary]) :: term

Remove tags from stream

split_shard/3

Specs:

  • split_shard(name :: iodata, shard :: iodata, new_starting_hash_key :: iodata) :: term

Split a shard

stream_records/1

Specs:

  • stream_records(stream_name :: iodata) :: term

Returns a stream of kinesis records NOTE: This stream is basically INFINITE, in that it runs until the shard it is reading from closes, which may be never. If you want it to take records until there are no more (at the moment), something like

Kinesis.stream_records("my-stream")
|> Enum.take_while(fn(val) -> !match?(%{"Data" => []}, val))

ought to do the trick.

The optional iterator_fun is a function that is called after every actual AWS request. Generally speaking you won’t need this, but it can be handy if you’re trying to prevent flooding. See Mix.Tasks.Kinesis.Tail.get_records/1 for an example.

stream_records/2

Specs:

  • stream_records(stream_name :: iodata, opts :: %{}) :: term
stream_records/3

Specs:

  • stream_records(stream_name :: iodata, opts :: %{}, iterator_fun :: Fun) :: term
stream_shards/1

Specs:

  • stream_shards(stream_name :: iodata :: %{}) :: term
  • stream_shards(stream_name :: iodata) :: term

Same as describe_stream/1,2 except the shards key is a stream and will automatically handle pagination Returns the normally shaped AWS response, except the Shards key is now a stream

stream_shards/1

Specs:

  • stream_shards(stream_name :: iodata :: %{}) :: term
  • stream_shards(stream_name :: iodata) :: term