Feedistiller

Provides functions to downloads enclosures of rss/atom feeds.

Features:

  • download multiple feeds at once and limit the number of downloads occurring at the same (globally or on per feed basis).
  • various filtering options:
  • content-type criteria
  • item name criteria
  • item date criteria

HTTPoison must be started to use Feedistiller functions.

Summary

Functions

Download enclosures of the given feed according to its settings

Download a set of feeds according to their settings

Download a set of feeds according to their settings, using the given semaphore to limit the number of simultaneous downloads

Functions

download_feed(feed, global_sem \\ nil)

Specs

download_feed(Feedistiller.FeedAttributes.t, Alambic.Semaphore.t | nil) ::
  :ok |
  {:error, String.t}

Download enclosures of the given feed according to its settings.

Attributes of the feed are:

  • url: the url of the feed. Redirect are auto followed.
  • destination: path for the downloaded files. Files are put in a subdirectory matching the feed name.
  • max_simultaneous_downloads: maximum number of simultaneous downloads for this file. Default is 3. Can be set to :unlimited for no limit.
  • filters: a set of filters to apply to the downloaded files:
  • limits: limits on the number of files to download:

    • to: download files up to this date (default is :latest)
    • from: download files from this date (default is :oldest)
    • max: download at most max files (default is :unlimited)
  • mime: a list of regex to apply to the ‘content-type’ field of the enclosure. Only ‘content-type’ passing those regex are downloaded.
  • name: a list of regex to apply to the name of the feed items. Only enclosure attached to names matching those regex are downloaded.
download_feeds(feeds)

Specs

download_feeds([Feedistiller.FeedAttributes.t]) :: :ok

Download a set of feeds according to their settings.

download_feeds(feeds, max)

Specs

download_feeds([Feedistiller.FeedAttributes.t], Alambic.Semaphore.t | nil) :: :ok
download_feeds([Feedistiller.FeedAttributes.t], integer) :: :ok

Download a set of feeds according to their settings, using the given semaphore to limit the number of simultaneous downloads.