funkspector v0.9.1 Funkspector.PageScraper
Provides a method to scrape an HTML page, given its URL.
Link to this section Summary
Functions
Fetches the given URL, follows redirections, and returns the data scraped from its HTML.
Link to this section Functions
Link to this function
scrape(original_url, options \\ %{})
Fetches the given URL, follows redirections, and returns the data scraped from its HTML.
Examples
iex> { :ok, data } = Funkspector.PageScraper.scrape("http://example.com")
iex> data.scheme
"http"
iex> data.host
"example.com"
iex> data.root_url
"http://example.com/"
iex> data.links.http.internal
[]
iex> data.links.http.external
["https://www.iana.org/domains/example"]
iex> data.links.non_http
[]
iex> { :ok, data } = Funkspector.PageScraper.scrape("http://github.com")
iex> data.original_url
"http://github.com"
iex> data.final_url
"https://github.com/"