Return Styles: Pseud0ch, Terminal, Valhalla, NES, Geocities, Blue Moon.

Pages: 1-

Writing a Scraper

Name: Anonymous 2011-07-04 1:07

1) Use Perl with LWP
2) Use FIOC with beautiful soup
3) Other [please specify]

Name: Anonymous 2011-07-04 3:29

Really depends on how big the project is. Half of my scrapers are so simple that I just use curl with sed and cut (yes, I get paid for that).

Name: Anonymous 2011-07-04 3:36

I'll take #1 for the win.

Name: Anonymous 2011-07-04 4:34

bump

Name: Anonymous 2011-07-04 5:47

I had to resort to iMacros when all else failed. But usually >>2
and Perl does it. And don't bother to try to scrape any sites from Google.

Name: Anonymous 2011-07-04 6:32

I'll recommend Python and LXML. Includes a CSS selector API, which is great when you don't want to dig too deep into enterprise scalable internet technologies such as XPath.

Writing a Scraper

1 Name: Anonymous 2011-07-04 1:07

2 Name: Anonymous 2011-07-04 3:29

3 Name: Anonymous 2011-07-04 3:36

4 Name: Anonymous 2011-07-04 4:34