Spidey v0.3.2 Spidey.Crawler View Source
Link to this section Summary
Types
The options passed to Spidey.Crawler.crawl/3
. Currently only allows the
specification of the filter to apply to crawled URLs, uses
Spidey.Filter.DefaultFilter
by default.
Functions
Crawls a given url synchronously through a supervision tree under the name of
crawler_name
. It accepts the filter to apply to crawled urls through the
filter
option.
Link to this section Types
Specs
crawl_options() :: [{:filter, module()}]
The options passed to Spidey.Crawler.crawl/3
. Currently only allows the
specification of the filter to apply to crawled URLs, uses
Spidey.Filter.DefaultFilter
by default.
Link to this section Functions
Specs
crawl(String.t(), atom(), crawl_options()) :: [String.t()]
Crawls a given url synchronously through a supervision tree under the name of
crawler_name
. It accepts the filter to apply to crawled urls through the
filter
option.