Return Styles: Pseud0ch, Terminal, Valhalla, NES, Geocities, Blue Moon. Entire thread

Bot Spoofing

Name: Anonymous 2010-09-01 14:26

Hi,
I've been experimenting with bypassing basic website security, and I've come across a fairly simple method of spoofing as a botnet.

I configured Firefox to show as a Googlebot by creating general.useragent.override with the simple string "Googlebot/2.1 Compatible: http://www.googlebot.com/bot.html".

Great; it works! Checked to make sure, then enjoyed browsing as a Googlebot.

Now I want to try bypassing sites with robots.txt
Is there any shortcut for this that I can try while still using Googlebot? And if not, is there another crawler with a string I can link to like this one which does not respect robots.txt in the first place?

Thanks.

Name: Anonymous 2010-09-01 23:26

I never thought anyone could think changing a user agent string would magically force your browser to obey an optional directive meant for a completely different class of internet user. People always manage to be dumber than I anticipate.

Newer Posts
Don't change these.
Name: Email:
Entire Thread Thread List