rDNS searching HTTP logs?

I have 1123629205 bytes of logs on my server.

I was wondering, what would be a decent script to run a rDNS request on each of these IPs, with a 3 second timeout, then sort them into a list of different extensions?
For example, if I want to find where any .gov, .mil, and .edu addresses are connecting to and from, what would be the best way to do this?

My logs are like this:
THEIR.IPv4.IP.HERE MYDOMAIN.TLD [29/Jun/2010:07:30:50 +0000] "GET /somethinglol HTTP/1.1" 301 0 "http://referrer.tld/" "useragent"

Thanks in advance.

>>1
my server
I can't understand what use a server is to you if you can't figure out how to do this.

>run a rDNS request on each of these IPs
Do you realize that amount of time spent and bandwidth for these requests? A million IPs(with 1 IP/sec ) will take 11 days.


__________________
Orbis terrarum delenda est

awk '{print $1}' access.log | sort | uniq >> ~/ips.log

Now go to ~/, and cat ips.log | rdns.pl >> rdnses.log

rdns.pl:
#!/usr/bin/perl
use constant TIMEOUT => 2;
$SIG{ALRM} = sub {die "timeout"};

while (<>) { s/^(\S+)/{print "\n\n$1 \n"; lookup($1)}/e; }
continue { print; }

sub lookup {
    my $ip = shift;
    return $ip unless $ip=~/\d+\.\d+\.\d+\.\d+/;
    unless (exists $CACHE{$ip}) {
        my @h = eval <<'END';
        alarm(TIMEOUT);
        my @i = gethostbyaddr(pack('C4',split('\.',$ip)),2);
        alarm(0);
        @i;
END
    $CACHE{$ip} = $h[0] || undef;
    }
    return $CACHE{$ip} || $ip;
}

chmod +x rdns.pl, and run those commands. It should give you a list of all the related rDNSes. You can then grep from rdnses.log for ".gov" or whatever you are looking for, copy that IP, and grep your un-filtered logs for it.

Be aware though, rDNSes are not required to be accurate. I could set my rDNS as "loli.fbi.gov", and you would never know the difference.

Have fun!

Love, !losPenises at /g/. (I have no idea why you cross-posted this thread here, though)

>>3
Why aren't you trippin' today?

flavor>>5
because YHBT.

>>5
That's where you are wrong. You think that you could easily post Anonymously because you see yourself as above Anonymous, that at any point you could choose to be on his level and that would be fine. In reality, you are below him. He is controlled and restrained enough to neither need nor see a need for credit, personalities in the course of mature programming discussion.



__________________
Orbis terrarum delenda est

Import the logs into a database.

SELECT DISTINCT IP FROM LOGS

Then write a script to query both DNS servers directly and the various sites out there that'll do it for you, so as to spread the load.

Enjoy your AIDS AND FAIL, /prog/

Enjoy your AIDS AND FAIL, /prog/

>>9
>>10
Enjoy your /DOUBLE POST/.

>>7
When he trips it is easier to expunge him, /brog/.

>>9
>>10
Why?

>>13
Go away.