Return Styles: Pseud0ch, Terminal, Valhalla, NES, Geocities, Blue Moon. Entire thread

Interesting Python Scripts

Name: Anonymous 2009-03-31 12:28

Hey,
I just want to hear some stories about Python and what you did with it. I'm especially interested in if you have ever used mechanize & html5lib (Or something similiar) to write some bots/scrapers etc. for webpages. I actually think forum-bots/spam-bots etc. are annoying but at the same time very interesting, especially when they have got a minimal AI or something comparable to it.

Please don't call me a troll or whatever, that was a real question and I would like to have serious answers and maybe some interesting code-snippets. (Of course I can't force you guys).

Name: Anonymous 2009-03-31 12:38

The original version of 4scrape1 was in Python. Rewrote the front-end in Haskell because Python is almost SLOW AS FUCK (specifically: embedding a Python interpreter into each Apache daemon made a mess of things, and the Python FastCGI bindings are under-documented nightmares).

The scraper is still in Python though, and just uses urllib2. Never used mechanize or html5lib.

SORRY GUYS HE ASKED AND I COULDN'T RESIST. LOVE, TARO.
                       
References: [1] http://suigintou.desudesudesu.org/4scrape/index

Newer Posts
Don't change these.
Name: Email:
Entire Thread Thread List