Snap.Bulk (Snap v0.2.1) View Source
Supports streaming bulk operations against a Snap.Cluster
.
Link to this section Summary
Functions
Performs a bulk operation.
Link to this section Functions
Specs
perform(Enumerable.t(), module(), String.t(), Keyword.t()) :: :ok | Snap.Cluster.error() | {:error, Snap.BulkError.t()}
Performs a bulk operation.
Takes an Enumerable
of action structs, where each struct is one of:
actions = [
%Snap.Bulk.Action.Create{_id: 1, doc: %{foo: "bar"}},
%Snap.Bulk.Action.Create{_id: 2, doc: %{foo: "bar"}},
%Snap.Bulk.Action.Create{_id: 3, doc: %{foo: "bar"}}
]
actions
|> Snap.Bulk.perform(Cluster, "index")
It chunks the Enumerable
into pages, and pauses between pages for
Elasticsearch to catch up. Uses Stream
under the hood, so you can lazily
feed it a stream of actions, such as out of an Ecto.Repo
to bulk load
documents from an SQL database.
If no errors occur on any page it returns :ok
. If any errors occur, on
any page, it returns {:error, %Snap.BulkError{}}
, containing a list of
the errors. It will continue to the end of the stream, even if errors
occur.
Options:
page_size
- defines the size of each page, defaulting to 5000 actions.page_wait
- defines wait period between pages in ms, defaulting to 15000ms.