Crawler v0.3.1 Crawler.Fetcher.Recorder
Records information about each crawl for internal use.
Link to this section Summary
Functions
Records information about each crawl for internal use
Stores page data in Crawler.Store.DB
for internal or external consumption
Link to this section Functions
Link to this function
record(opts)
Records information about each crawl for internal use.
Examples
iex> Recorder.record(url: "url1", depth: 2)
{:ok, %{depth: 3, url: "url1"}}
iex> Recorder.record(url: "url2", depth: 2)
iex> Store.find("url2")
%Page{url: "url2"}
Link to this function
store_page(body, opts)
Stores page data in Crawler.Store.DB
for internal or external consumption.