Crawly v0.2.0 Crawly View Source

Crawly is a fast high-level web crawling & scraping framework for Elixir.

Link to this section Summary

Functions

Fetches a given url. This function is mainly used for the spiders development when you need to get individual pages and parse them

Link to this section Functions

Link to this function

fetch(url, headers \\ [], options \\ []) View Source
fetch(url, headers, options) :: HTTPoison.Response.t()
when url: binary(), headers: [], options: []

Fetches a given url. This function is mainly used for the spiders development when you need to get individual pages and parse them