funkspector v0.0.1 Funkspector.Scraper
Provides a method to scrape a given URL.
Summary
Functions
Fetches the given URL, follows redirections, and returns the data scraped from its HTML
Functions
Fetches the given URL, follows redirections, and returns the data scraped from its HTML.
Examples
iex> { :ok, data } = Funkspector.scrape("http://jaimeiniesta.com")
iex> data.scheme
"http"
iex> data.host
"jaimeiniesta.com"
iex> data.root_url
"http://jaimeiniesta.com/"
iex> data.links.http.internal
["http://jaimeiniesta.com/",
"http://jaimeiniesta.com/about/",
"http://jaimeiniesta.com/archive/",
"http://jaimeiniesta.com/portfolio/",
"http://jaimeiniesta.com",
"http://jaimeiniesta.com/articles/questions-about-getting-into-freelancing/",
"http://jaimeiniesta.com/articles/building-a-disqus-recent-comments-widget-with-javascript/",
"http://jaimeiniesta.com/articles/tips-for-a-new-rails-developer/",
"http://jaimeiniesta.com/articles/fifteen-servers/",
"http://jaimeiniesta.com/atom.xml"]
iex> data.links.http.external
["http://jekyllrb.com",
"http://mademistakes.com/so-simple/",
"http://twitter.com/jaimeiniesta",
"http://facebook.com/jaime.iniesta.7",
"http://plus.google.com/+jaimeiniesta",
"http://linkedin.com/in/jaimeiniesta",
"http://instagram.com/jaimeiniesta",
"http://www.flickr.com/photos/jaimeiniesta", "http://github.com/jaimeiniesta"]
iex> data.links.non_http
["mailto:jaimeiniesta@gmail.com"]
iex> { :ok, data } = Funkspector.scrape("http://github.com")
iex> data.original_url
"http://github.com"
iex> data.final_url
"https://github.com/"