crawlie v0.1.1 Crawlie
Summary
Functions
Crawls the urls provided in source
, using the Crawlie.ParserLogic
provided
in parser_logic
Functions
Crawls the urls provided in source
, using the Crawlie.ParserLogic
provided
in parser_logic
.
The options
are used to tweak the crawler’s behaviour. You can use most of
the options for HttPoison,
as well as Crawlie specific options.
Crawlie options
:http_client
- module implementing theCrawlie.HttpClient
behaviour to be used to make the requests.:mock_client_fun
- If you’re using theCrawlie.HttpClient.MockClient
, this would be theurl -> {:ok, body :: String.t} | {:error, term}
function simulating making the requests.