crawlie v1.0.0 API Reference

Modules

The simple Elixir web crawler

Defines the struct representing a url’s state in the system

Wraps the Erlang pqueue :pqueue, :pqueue2, :pqueue3 and :pqueue4 modules functionality for easier use in Elixir and easier swapping of the implementations

Defines the struct representing a page retrieved by the http client

Tracks the crawling statistics of a particular crawling session