Crawly v0.2.0 Crawly.Middlewares.UniqueRequest View Source

Avoid scheduling requests for the same pages.

Link to this section Summary

Link to this section Functions