Crawler v1.0.0 Crawler.Fetcher.Recorder

Records information about each crawl for internal use.

Link to this section Summary

Functions

Records information about each crawl for internal use

Stores page data in Crawler.Store.DB for internal or external consumption

Link to this section Functions

Records information about each crawl for internal use.

Examples

iex> Recorder.record(url: "url1", depth: 2)
{:ok, %{depth: 3, url: "url1"}}

iex> Recorder.record(url: "url2", depth: 2)
iex> Store.find("url2")
%Page{url: "url2"}
Link to this function store_page(body, opts)

Stores page data in Crawler.Store.DB for internal or external consumption.