Kreuzcrawl.CrawlPageResult (kreuzcrawl v0.3.0-rc.37)

Copy Markdown

The result of crawling a single page during a crawl operation.

Summary

Types

t()

The result of crawling a single page during a crawl operation.

Types

t()

@type t() :: %Kreuzcrawl.CrawlPageResult{
  body_size: non_neg_integer(),
  browser_used: boolean(),
  content_type: String.t() | nil,
  depth: non_neg_integer(),
  detected_charset: String.t() | nil,
  downloaded_document: map() | nil,
  extracted_data: String.t() | nil,
  extraction_meta: map() | nil,
  feeds: [map()],
  html: String.t() | nil,
  images: [map()],
  is_pdf: boolean(),
  json_ld: [map()],
  links: [map()],
  markdown: map() | nil,
  metadata: map(),
  normalized_url: String.t() | nil,
  status_code: non_neg_integer(),
  stayed_on_domain: boolean(),
  url: String.t() | nil,
  was_skipped: boolean()
}

The result of crawling a single page during a crawl operation.