Sputnik v0.2.0 Page View Source

This module fetches and parses a given URL.

Link to this section Summary

Functions

Asyncronously fetches a given URL. It parses the body and sends back to Queue the list of links to follow. It parses the body and counts how many CSS selectors there are in the page

Link to this section Functions

Link to this function start(url, query, queue_pid) View Source

Asyncronously fetches a given URL. It parses the body and sends back to Queue the list of links to follow. It parses the body and counts how many CSS selectors there are in the page

Parameters

  • url: the initial URL to crawl
  • query: list of valid CSS selectors as strings
  • queue_pid: the pid which will receive the output