Xarn

It's been a long time since the last discussion on this topic.

Hello Xarn

>>41
You can test it yourself, but it took ~16 min for a full scrape from scratch, as opposed to ~2 hours with Xarn's scraper¹ (and at that time there were more posts to scrape).
It uses (by default) 16 worker threads and uses the JSON interface (which Xarn might have not been aware of) whenever it can. It also downloads only the missing posts (post selection also works with the JSON interface).
And it has a nice progress bar.

_{1: http://cairnarvon.rotahall.org/2008/12/25/χριστούγεννα/}