Spidey v0.1.0 Spidey View Source
Spidey is a basic web crawler which runs through all the links of a same domain and outputs them in a simple text sitemap format.
Link to this section Summary
Functions
Crawls a website for all the same-domain urls, returning a list.
Crawls a website for all the sam-domain urls and Saves the list of urls to file
Link to this section Functions
Crawls a website for all the same-domain urls, returning a list.
Crawls a website for all the sam-domain urls and Saves the list of urls to file