funkspector v0.9.1 Funkspector.PageScraper

Provides a method to scrape an HTML page, given its URL.

Link to this section Summary

Functions

Fetches the given URL, follows redirections, and returns the data scraped from its HTML.

Link to this section Functions

Link to this function

scrape(original_url, options \\ %{})

Fetches the given URL, follows redirections, and returns the data scraped from its HTML.

Examples

iex> { :ok, data } = Funkspector.PageScraper.scrape("http://example.com")
iex> data.scheme
"http"
iex> data.host
"example.com"
iex> data.root_url
"http://example.com/"
iex> data.links.http.internal
[]
iex> data.links.http.external
["https://www.iana.org/domains/example"]
iex> data.links.non_http
[]

iex> { :ok, data } = Funkspector.PageScraper.scrape("http://github.com")
iex> data.original_url
"http://github.com"
iex> data.final_url
"https://github.com/"