Resource limits

Copy Markdown

FeedReader parses XML incrementally, but it retains feed metadata, the current item, pending stream events, and—when collecting—a list of items. Configure limits for the resources that matter to your workload. These limits help avoid memory or CPU spikes while parsing and make resource use more predictable across feeds. Invalid values generally fall back to their defaults; the options below accept non-negative integers unless noted.

XML parsing limits

OptionDefaultBounds
:max_detection_bytes65,536 bytesPrefix inspected to detect a format; must be positive.
:chunk_size65,536 bytesXML input chunks and SAX character-data callback sizing; must be positive.
:max_xml_events1,000,000SAX events processed.
:max_xml_depth256Nested XML element depth.
:max_attributes_per_element128Attributes on one element.
:max_attribute_bytes65,536 bytesCombined name and value size of an individual attribute.
:max_feed_metadata_bytes1 MiBRetained feed metadata.
:max_item_bytes4 MiBRetained content for the current item.
:max_queue_bytesDefaults to :max_item_bytesSerialized size of one pending event.

Zero is a valid limit for the non-negative resource budgets. A zero-byte metadata or item budget, for example, rejects retained data in that scope. Invalid XML-limit values use defaults. An invalid :max_detection_bytes disables detection; supply a valid explicit :format to parse in that case.

:max_queue_bytes bounds each serialized feed or item event independently. The limit also guards normalized media output after URL resolution, so a short root-relative source URL can expand against a long feed base. If the media budget is exceeded, collection returns {:error, {:resource_limit, :max_queue_bytes, limit}}; streaming emits the same reason as a terminal error event. This event-size budget is separate from the XML input limits above. Media candidates are normalized incrementally against the budget; the final serialized-size check remains authoritative.

Application configuration

Set resource budgets globally with :feed_reader, :options and override them for individual calls when needed:

config :feed_reader, :options,
  max_items: 1_000,
  max_xml_depth: 128,
  max_compressed_bytes: 10_000_000

FeedReader.parse_file("path/to/feed.xml", max_items: 100)

Per-call values take precedence over application config, which takes precedence over built-in defaults. The application value must be a keyword list; malformed values are ignored. :feed_url, :previous_etag, :previous_checksum, and :item_filter are request-specific and cannot be set globally. See FeedReader for the complete option-default reference.

Example for stricter parsing budgets:

FeedReader.stream_file("path/to/feed.xml",
  max_xml_events: 100_000,
  max_xml_depth: 64,
  max_attributes_per_element: 32,
  max_feed_metadata_bytes: 256_000,
  max_item_bytes: 1_000_000,
  max_queue_bytes: 1_000_000
)

Limit failures use the form {:error, {:resource_limit, option, limit}} from collection APIs. In a stream, parsing errors are delivered as a terminal {:error, reason} event.

Remote download limits

Remote APIs accept these additional budgets:

OptionDefaultBounds
:max_compressed_bytes50 MiBRaw response bytes received.
:max_decoded_bytes50 MiBBytes after gzip/deflate decoding.
:max_temp_bytes50 MiBDecoded bytes staged in the temporary file.
:max_expansion_ratio100Decoded-to-compressed byte ratio.

The limits are checked while chunks are consumed. A failure removes any temporary file created for the download. Content encodings supported by the default pipeline are identity, gzip, and deflate; unsupported or stacked encodings are rejected.

Collection limits

parse_file/2, parse_binary/2, parse_stream/2, and fetch_and_parse/2 collect all accepted items by default. Supply a positive :max_items to stop parsing as soon as that many accepted items have been collected. The optional one-argument :item_filter rejects items before they are counted and retained. When the cap is reached, the result has item_limit_reached?: true; it may be only a prefix of the feed, and the rest of the feed is not parsed or validated.

The item limit bounds retained item count, not input bytes or network transfer. For remote feeds, the full response is downloaded before parsing. parse_stream/2 also stages all input chunks before parsing. Without a cap, memory use grows with the collected items. Use stream_file/2 or stream/2 to process large feeds incrementally without collecting all items in memory. Combine any collection cap with download and XML limits when you need those separate guarantees.

Verified APIs additionally accept :max_parse_passes, defaulting to 5, to bound validation and event-delivery passes. An invalid value uses the default.