funkspector v0.7.1 Funkspector.PageScraper

Provides a method to scrape a page, given its URL.

Summary

Functions

Fetches the given URL, follows redirections, and returns the data scraped from its HTML

Functions

scrape(original_url, options \\ %{})

Fetches the given URL, follows redirections, and returns the data scraped from its HTML.

Examples

iex> { :ok, data } = Funkspector.PageScraper.scrape("http://example.com")
iex> data.scheme
"http"
iex> data.host
"example.com"
iex> data.root_url
"http://example.com/"
iex> data.links.http.internal
[]
iex> data.links.http.external
["http://www.iana.org/domains/example"]
iex> data.links.non_http
[]

iex> { :ok, data } = Funkspector.PageScraper.scrape("http://github.com")
iex> data.original_url
"http://github.com"
iex> data.final_url
"https://github.com/"