Return Styles: Pseud0ch, Terminal, Valhalla, NES, Geocities, Blue Moon. Entire thread

Circumvent block on page downloading

Name: Anonymous 2013-12-13 16:39

I'm trying to use PHP to let users enter URLs of a specific domain and return data about the content of that page, but the domain blocks programs that try to "download pages from this site automatically." Any ideas on how I might circumvent this?

Name: Anonymous 2013-12-13 16:56

Easy.

Fake Googlebot's user agent. Webmasters of shitty websites don't block it.

Newer Posts
Don't change these.
Name: Email:
Entire Thread Thread List