Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I would look at "delta-scraping" as one of the things that is easier with scrapy. There is an existing extension for it: https://github.com/scrapinghub/scrapylib/blob/master/scrapyl... and lots of support and help from the community.

Admittedly writing this from scratch requires learning the framework, but being able to share and reuse code is a huge win (disclaimer: I'm a Scrapy contributor).



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: