Crawler v1.1.1 Crawler.Fetcher.Recorder

Records information about each crawl for internal use.

Link to this section Summary

Functions

Records information about each crawl for internal use.

Stores page data in Crawler.Store.DB for internal or external consumption.

Link to this section Functions

Records information about each crawl for internal use.

Examples

iex> Recorder.record(url: "url1", depth: 2)
{:ok, %{depth: 3, url: "url1"}}

iex> Recorder.record(url: "url2", depth: 2)
iex> Store.find("url2")
%Page{url: "url2"}
Link to this function

store_page(body, opts)

Stores page data in Crawler.Store.DB for internal or external consumption.