Streaming and verified parsing
Copy MarkdownStreaming APIs return a %FeedReader.Session{} containing a one-pass
events enumerable. Events are delivered in order: one {:feed, feed} event
followed by normalized {:item, item} events. A parser error is emitted as a
terminal {:error, reason} event. Errors that happen before a session can be
created, such as an unknown format or a failed remote request, are returned
directly as {:error, reason}.
The feed event includes a single feed-metadata snapshot. RSS and Atom include eligible channel/feed media seen before the first item; RDF uses the channel available when the feed event is emitted. Later feed-level media does not update that event. Item media is emitted with its item in source encounter order. These normalized feed and item values are the same ones returned by collection APIs.
Streaming APIs do not enforce :max_items; collection APIs such as
parse_file/2, parse_stream/2, and fetch_and_parse/2 collect all accepted
items by default and can use the option as an early-stop cap. A stream emits
items as the consumer enumerates them. Stop enumeration in the consumer to limit
how many events are processed.
Stream a local file
{:ok, session} = FeedReader.stream_file("path/to/feed.xml")
session.events
|> Enum.each(fn
{:feed, feed} -> save_feed_metadata(feed)
{:item, item} -> process_item(item)
{:error, reason} -> handle_parse_error(reason)
end)The local file remains caller-owned, even if enumeration stops early. The
parser runs on demand and only advances as events are requested. :chunk_size
controls input chunks and SAX character-data callback sizing.
Stream a remote URL
{:ok, session} = FeedReader.stream("https://example.com/feed.xml")
try do
session.events
|> Enum.take(20)
|> Enum.each(&process_event/1)
after
FeedReader.close(session)
endEnum.take/2 limits events, including the initial feed-metadata event; it does
not count only item events.
Remote sessions include a fetch record. The remote response is downloaded,
decoded, checksummed, and staged in an owned temporary file before
stream/2 returns. Parsing the staged file is lazy; the network download is
not. Fully consuming events removes the temporary file. If enumeration stops
early, raises, or otherwise may not finish, call FeedReader.close/1 in an
after block. Closing is safe after completion and can be repeated.
An ETag 304 or a matching previous checksum returns
{:ok, :not_modified} or {:ok, :not_changed} respectively, instead of a
session.
Collect input chunks into a result
parse_stream/2 is different from stream_file/2: it takes XML chunks but
collects the result into memory, optionally stopping parsing at :max_items.
Chunks must be binaries. The chunks are staged to a temporary file before
parsing, so use stream_file/2 or stream/2 to process large feeds
incrementally.
chunks = File.stream!("path/to/feed.xml", 16_384, [:raw])
{:ok, result} = FeedReader.parse_stream(chunks)Chunk input is staged in a library-owned temporary file and cleaned up on
success and failure. If a supplied item cap is reached, the result's
item_limit_reached? field is true, and later parser errors are not observed.
Use stream_file/2 when you want to process events without collecting all items.
Validate before exposing events
stream_file_verified/2 and stream_verified/2 validate the feed before
returning an event session. This is useful when consumers must not see any
events unless parsing succeeds. The validation pass adds work; delivery still
parses the source to emit events. :max_parse_passes bounds validation and
delivery passes and defaults to 5 (a value below 2 cannot allow both
passes).
{:ok, session} = FeedReader.stream_file_verified("path/to/feed.xml")
Enum.each(session.events, &process_event/1)Verified remote sessions have the same temporary-file ownership rules as
stream/2: close them in after when enumeration may end early. Like the
other parser APIs, parsing stops when the feed root closes; trailing content
after a complete root is not parsed as part of the feed.