Spidey v0.3.2 Spidey.Filter.DefaultFilter View Source
An implementation of the Spidey.Filter
behaviour which:
- Transforms relative urls to absolute urls
- Strips the query parameters of all urls, to simplify unicity.
- Strips the trailing slashes of all urls.
- Rejects all urls from a different domain than the seed's.
- Rejects invalid urls
- Reject static resources based on different criteria such as wordpress paths and file type.
This behaviour requires the option :seed
.
Link to this section Summary
Link to this section Functions
Specs
process_relative_urls(Enumerable.t(), String.t()) :: Enumerable.t()
Specs
reject_invalid_urls(Enumerable.t()) :: Enumerable.t()
Specs
reject_non_domain_urls(Enumerable.t(), String.t()) :: Enumerable.t()
Specs
reject_static_resources(Enumerable.t()) :: Enumerable.t()
Specs
strip_query_params(Enumerable.t()) :: Enumerable.t()
Specs
strip_trailing_slashes(Enumerable.t()) :: Enumerable.t()