funkspector v0.9.1 Funkspector

Funkspector is a web scraper that lets you extract data from web pages.

Link to this section Summary

Link to this section Functions

Link to this function

default_options()

Link to this function

page_scrape(url, options \\ %{})

Convenience method, this is just a shortcut for Funkspector.PageScraper.scrape/1.

Examples

iex> { :ok, data } = Funkspector.page_scrape("http://example.com")
iex> data.host
"example.com"
Link to this function

sitemap_scrape(url, options \\ %{})

Convenience method, this is just a shortcut for Funkspector.SitemapScraper.scrape/1.

Examples

iex> { :ok, data } = Funkspector.sitemap_scrape("https://rocketvalidator.com/sitemap.xml")
iex> length data.locs
527
iex> [ first | _ ] = data.locs
iex> first
"https://rocketvalidator.com/"
Link to this function

text_sitemap_scrape(url, options \\ %{})

Convenience method, this is just a shortcut for Funkspector.TextSitemapScraper.scrape/1.

Examples

iex> { :ok, data } = Funkspector.text_sitemap_scrape("https://rocketvalidator.com/sitemap.txt")
iex> length data.lines
527
iex> [ first | _ ] = data.lines
iex> first
"https://rocketvalidator.com/"