FeedReader
Copy MarkdownFeedReader fetches and parses RSS, Atom, and RDF feeds into normalized Elixir data. It detects formats automatically, recovers from a small set of common XML defects, and offers collection with an optional item cap as well as lazy event streams.
[!NOTE] The canonical repository is hosted on Codeup. Copies on GitHub, GitLab, and other forges are mirrors for broader distribution; project development and updates are maintained in the canonical repository.
Installation
Add FeedReader to your Mix project's dependencies:
defp deps do
[
{:feed_reader, "~> 0.2.0"}
]
endQuick start
Pass XML directly, or use parse_file/2 for a local file. Collection APIs gather
all accepted items by default; an optional :max_items cap stops parsing early:
{:ok, result} = FeedReader.parse_file("path/to/feed.xml")
result.feed.title
Enum.map(result.items, & &1.title)For a remote URL, fetch_and_parse/2 downloads and collects the feed. You can
set :max_items to stop parsing after a given number of accepted items; when
that happens, result.item_limit_reached? is true. The complete response is
downloaded before parsing starts. The default HTTP transport follows redirects
automatically, up to five; the final response URL is available in
result.fetch.final_url.
{:ok, result} =
FeedReader.fetch_and_parse("https://example.com/feed.xml")
result.feed.title
result.fetch.etagApplication configuration
Set stable options in your application config. For example, configure an optional collection cap alongside other defaults:
config :feed_reader, :options,
max_items: 500,
max_xml_depth: 128,
http_retry_attempts: 3,
formats: [MyApp.NoticeFormat],
http_client: MyApp.FeedTransportThe configured cap applies to collection calls that omit :max_items; per-call
values override it. If no cap is configured, collection gathers all accepted
items:
{:ok, result} = FeedReader.parse_file("path/to/feed.xml")
{:ok, capped_result} = FeedReader.parse_file("path/to/feed.xml", max_items: 100)Use stream_file/2 or stream/2 to process large feeds incrementally rather
than collecting all items in memory. parse_stream/2 still collects its result;
its input chunks are staged before parsing.
Request-specific options such as :feed_url, :previous_etag,
:previous_checksum, and :item_filter must be supplied per call. See the
resource limits guide and FeedReader API docs for
defaults and details. :formats adds custom formats after the built-ins;
:http_client selects a custom transport. Custom transports may define and read
their own additional options.
Architecture
The public API coordinates a small pipeline:
- Remote requests use a pluggable transport. The downloader consumes raw response chunks, decodes supported content encodings, enforces download budgets, computes a decoded-body checksum, and stages the result in an owned temporary file.
- The parser inspects a bounded prefix to select RSS, Atom, RDF, or a supplied custom format. An explicit format can be selected when detection is unsuitable.
- Streaming XML recovery normalizes supported encoding and markup defects
before Saxy parses the input. Format handlers normalize metadata and items
into
FeedReader.FeedandFeedReader.Itemvalues. - The parser emits
{:feed, feed}and{:item, item}events. Streaming APIs expose those events lazily; collection APIs gather all accepted items by default, or stop at an optional item cap.
Remote streaming downloads before returning its session; event parsing then
reads the staged file lazily. A completed stream releases the temporary file.
If remote enumeration stops early, close the session with FeedReader.close/1.
Local sessions read caller-owned files and never remove them.
Guides
- Parsing and change detection — files, binaries, streams, remote URLs, and unchanged-source checks.
- Streaming and verified parsing — event lifecycle, cleanup, backpressure, and validation passes.
- Resource limits — parser, download, and collection budgets.
- Results and normalized data — result structs, events, normalized fields, and common outcomes.
- Customization — format detection, custom formats, transports, and HTTP behavior.
- XML recovery — supported repairs and their limits.
See the FeedReader module documentation for the complete API and option
reference.
Contributing
Install Elixir 1.20.4 with Erlang/OTP
29.1.2, as specified in .tool-versions. Then fetch dependencies and run the
project checks:
mix deps.get
mix precommit
mix credo --strict
Create a branch for your change and submit it as a pull request. mix precommit
compiles with warnings treated as errors, checks formatting, and runs the tests.