Funkspector.Utils (funkspector v1.4.0)
Common functionality for the scrapers.
Summary
Functions
Given a collection of URLs and a base URL, absolutifies the relative links.
Returns a boolean telling if the URL seems valid according to the regexp.
Functions
Given a collection of URLs and a base URL, absolutifies the relative links.
Examples
iex> Funkspector.Utils.absolutify(["javascript:alert(hi)", "/faqs?section=legal", "/about", "http://example.com/faqs"], "http://example.com")
["javascript:alert(hi)", "http://example.com/faqs?section=legal", "http://example.com/about", "http://example.com/faqs"]
Returns a boolean telling if the URL seems valid according to the regexp.
Examples
iex> Funkspector.Utils.valid_url?("https://example.com") true
iex> Funkspector.Utils.valid_url?("joe@example.com") false
iex> Funkspector.Utils.valid_url?(nil) false
iex> Funkspector.Utils.valid_url?(" ") false