Monks, after doing all recommended changes I was able to process 3,000 records per second (for a total of 259,200,000 records a day).
All changes had some positive effect, but the one that had the most dramatic effect was moving to Geo::IP. As it was mentioned on the thread by a few of you guys, the relational database was certainly not the correct place to do this operation.
The finalized code looks like this now:
sub Obtain_GeoIP {
$GeoIP_resolved++;
my ($IP) = @_;
if($IP=~/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/ &&(($1<
+=255 && $2<=255 && $3<=255 &&$4<=255 )))
{
my @sip = split(/\./,$IP);
if (($sip[0] == "10") || ($sip[0] == "192" && $sip[1] == "168"
+) || (($sip[0] == "172" && $sip[1] >= "16") && ($sip[0] == "172" && $
+sip[1] <= "31")))
{
return "Internal Network";
} else {
if ( exists $geoip_hash{IP} ) { my $country_hash = $ge
+oip_hash{IP}; return $country_hash; }
my $country = $gi->country_name_by_addr($IP);
$geoip_hash{$IP} = $country;
return $country;
}
} else {
print EFILE "IP address sent to Obtain_GeoIP routine w
+as incorrect: $IP - $counter\n";
}
}
Thank you very much for everybody's help! | [reply]
[d/l] |