Coffrify.Runtime.Paginate (Coffrify v0.9.0)

Stream-based pagination — works for every { data: [...], has_more, next_cursor } list endpoint.

Example

stream =
  Coffrify.Runtime.Paginate.paginate(fn opts ->
    {:ok, page} = Coffrify.Resources.Transfers.list(client, opts)
    page
  end, page_size: 100, mode: :cursor)

stream
|> Stream.filter(& &1["scan_status"] == "infected")
|> Enum.take(10)

Summary

Types

fetch_fun()

Per-page fetcher receives [limit, offset, cursor] keyword.

opts()

Options.

page()

A page returned by the fetcher.

Functions

collect(stream)

Convenience: collect a paginated stream into a list. Be careful with large datasets.

paginate(fetch_page, opts \\ [])

Build a lazy Stream that yields items one by one.

Types

fetch_fun()

@type fetch_fun() :: (keyword() -> page())

Per-page fetcher receives [limit, offset, cursor] keyword.

opts()

@type opts() :: [
  page_size: pos_integer(),
  limit: pos_integer() | :infinity,
  offset: non_neg_integer(),
  mode: :cursor | :offset
]

Options.

page()

@type page() :: %{
  optional(:data) => list(),
  optional(:transfers) => list(),
  optional(:entries) => list(),
  optional(:has_more) => boolean(),
  optional(:next_cursor) => String.t() | nil,
  optional(:total) => non_neg_integer()
}

A page returned by the fetcher.

Functions

collect(stream)

@spec collect(Enumerable.t()) :: list()

Convenience: collect a paginated stream into a list. Be careful with large datasets.

paginate(fetch_page, opts \\ [])

@spec paginate(fetch_fun(), opts()) :: Enumerable.t()

Build a lazy Stream that yields items one by one.

Options:

:page_size (default 100) — how many items per HTTP request.
:limit — hard cap on the total number of items (default :infinity).
:offset — initial offset (offset mode only).
:mode — :cursor (default) or :offset.