Kreuzcrawl.ScrapeResult (kreuzcrawl v0.3.0-rc.43)

Copy Markdown

The result of a single-page scrape operation.

Summary

Types

t()

The result of a single-page scrape operation.

Types

t()

@type t() :: %Kreuzcrawl.ScrapeResult{
  assets: [map()],
  auth_header_sent: boolean(),
  body_size: non_neg_integer(),
  browser: map() | nil,
  browser_used: boolean(),
  content_type: String.t() | nil,
  crawl_delay: non_neg_integer() | nil,
  detected_charset: String.t() | nil,
  downloaded_document: map() | nil,
  extracted_data: String.t() | nil,
  extraction_meta: map() | nil,
  feeds: [map()],
  final_url: String.t() | nil,
  html: String.t() | nil,
  images: [map()],
  is_allowed: boolean(),
  is_pdf: boolean(),
  js_render_hint: boolean(),
  json_ld: [map()],
  links: [map()],
  markdown: map() | nil,
  metadata: map(),
  nofollow_detected: boolean(),
  noindex_detected: boolean(),
  response_meta: map() | nil,
  status_code: non_neg_integer(),
  was_skipped: boolean(),
  x_robots_tag: String.t() | nil
}

The result of a single-page scrape operation.