Marqeta.Stream (marqeta v1.0.0)

Lazy streaming over paginated Marqeta list endpoints.

Fetches the next page only when the stream consumer exhausts the current one. No prefetching or buffering occurs.

Examples

# Stream all users (auto-paginates)
Marqeta.Stream.stream(&Marqeta.Users.list/1, %{count: 100})
|> Stream.filter(& &1["status"] == "ACTIVE")
|> Enum.to_list()

# Take only the first 50 items
Marqeta.Stream.stream(fn p -> Marqeta.Cards.list_by_user("tok", p) end)
|> Enum.take(50)

# Collect all pages into one list
{:ok, all} = Marqeta.Stream.all(&Marqeta.Users.list/1)

Summary

Functions

all(list_fn, params \\ %{})

Collects all pages into a single list synchronously.

stream(list_fn, params \\ %{}, opts \\ [])

Returns a lazy Enumerable that auto-paginates using list_fn.

Functions

all(list_fn, params \\ %{})

@spec all((map() -> {:ok, map()} | {:error, term()}), map()) ::
  {:ok, [map()]} | {:error, term()}

Collects all pages into a single list synchronously.

Returns {:ok, [item]} or {:error, error}. Prefer stream/3 for large datasets.

stream(list_fn, params \\ %{}, opts \\ [])

@spec stream(
  (map() -> {:ok, map()} | {:error, term()}),
  map(),
  keyword()
) :: Enumerable.t()

Returns a lazy Enumerable that auto-paginates using list_fn.

list_fn must be a 1-arity function accepting a params map and returning {:ok, page} or {:error, error}.

Options

:raise_on_error — raise Marqeta.Error instead of stopping the stream silently on error. Default: false.