crawlie v0.3.0 Crawlie.Stage.UrlManager.State
Summary
Functions
Marks the url as “already visited” in the state
Checks if the url was already visited by the crawler
Types
Functions
finished_processing(state, url)
finished_processing(Crawlie.Stage.UrlManager.State.t, String.t) :: Crawlie.Stage.UrlManager.State.t
started_processing(state, url)
started_processing(Crawlie.Stage.UrlManager.State.t, String.t) :: Crawlie.Stage.UrlManager.State.t
take_pages(state, count)
take_pages(Crawlie.Stage.UrlManager.State.t, integer) :: {Crawlie.Stage.UrlManager.State.t, [Crawlie.Page.t]}
Checks if the url was already visited by the crawler