Kreuzcrawl.CrawlEngineHandle (kreuzcrawl v0.3.0-rc.37)

Copy Markdown

Opaque handle to a configured crawl engine.

Constructed via [create_engine] with an optional [CrawlConfig]. Default implementations for all pluggable components are used internally.

Summary

Types

t()

Opaque handle to a configured crawl engine.

Functions

Stream a multi-URL crawl, yielding [CrawlEvent]s across all seeds.

Stream a single-URL crawl, yielding [CrawlEvent]s as pages are processed.

Types

t()

@type t() :: %Kreuzcrawl.CrawlEngineHandle{ref: reference()}

Opaque handle to a configured crawl engine.

Functions

batch_crawl_stream(obj, req)

Stream a multi-URL crawl, yielding [CrawlEvent]s across all seeds.

crawl_stream(obj, req)

Stream a single-URL crawl, yielding [CrawlEvent]s as pages are processed.