in reply to Slow response on DBI MySQL query

Besides the Gep::IP module, there is also the IP::Country module for "fast lookup of country codes from IP addresses".

I have not used either of them, but they both will do what you need.

I would also use the is_ipv4() and is_private_ipv4() functions from the Data::Validate::IP module to handle the validation of the IP.

Also, instead of manually converting the IP to an int, I'd use the built-in mysql function inet_aton() in the query.

Replies are listed 'Best First'.
Re^2: Slow response on DBI MySQL query
by Fortificus (Novice) on Mar 24, 2015 at 14:52 UTC
    Fishmonger, lots of really good recommendations! It will take me a little bit to change the script with all of them, and then I will come back to post the speed improvement of the recommendations. Thank you very much!
[reply]
      Monks, after doing all recommended changes I was able to process 3,000 records per second (for a total of 259,200,000 records a day). All changes had some positive effect, but the one that had the most dramatic effect was moving to Geo::IP. As it was mentioned on the thread by a few of you guys, the relational database was certainly not the correct place to do this operation. The finalized code looks like this now: 
      sub Obtain_GeoIP {
        $GeoIP_resolved++;
        my ($IP) = @_;
        if($IP=~/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/ &&(($1<
+=255  && $2<=255 && $3<=255  &&$4<=255 )))
        {
        my @sip = split(/\./,$IP);
        if (($sip[0] == "10") || ($sip[0] == "192" && $sip[1] == "168"
+) || (($sip[0] == "172" && $sip[1] >= "16") && ($sip[0] == "172" && $
+sip[1] <= "31")))
        {
                return "Internal Network";
        } else {
                if ( exists $geoip_hash{IP} ) { my $country_hash = $ge
+oip_hash{IP}; return $country_hash; }
                my $country = $gi->country_name_by_addr($IP);
                $geoip_hash{$IP} = $country;
                return $country;
        }
        } else {
                print EFILE "IP address sent to Obtain_GeoIP routine w
+as incorrect: $IP - $counter\n";
        }
}

      [download]
      Thank you very much for everybody's help!
[reply]
[d/l]

In Section Seekers of Perl Wisdom