Crawler v0.3.1 Crawler.Dispatcher

Dispatches requests to a queue for crawling.

Link to this section Summary

Functions

Takes the request argument which is a tuple containing either

Link to this section Functions

Link to this function dispatch(request, opts)

Takes the request argument which is a tuple containing either:

  • {_, link, _, url} when it’s a link that got transformed into a URL
  • {_, url} when it’s a URL already

And issues Crawler.crawl/2 to initiate the crawl.