View Source OffBroadway.Elasticsearch.SearchAfterStrategy (OffBroadwayElasticsearch v0.1.0)
Implementation of OffBroadway.Elasticsearch.Strategy
that's able to fetch
more than 10,000
results per request by using Elasticsearch's search_after
feature. This strategy is best suited for a single producer and the
search query must be sorted in some way. For concurrent searching, see
OffBroadway.Elasticsearch.SliceStrategy
Example
defmodule MyBroadway do
use Broadway
def start_link(_opts) do
Broadway.start_link(__MODULE__,
...
producer: [
module: {
OffBroadway.Elasticsearch.Producer,
[
host: "http://localhost:9200",
index: "my-index",
strategy: :search_after, # <- Select the 'search_after' strategy
search: search() # <- Provide a search query
]
},
concurrency: 1 # <- Best
],
...
)
end
def search do
%{
query: %{
match_all: %{}
},
sort: %{ # <- Must be present so Elasticsearch returns the `search_after` keys/values
created_at: "asc",
_id: "asc"
}
}
end
end
See Elasticsearch docs: Search After
Summary
Types
@type broadway_state() :: OffBroadway.Elasticsearch.broadway_state()
@type document() :: OffBroadway.Elasticsearch.document()