Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello
I have a list of IP addresses. In another file, I have another list of IP addresses, and the country from which they originate. I have keyed the array off the IP address.
What I need to do is take the IP in the first file, compare it to the second file, and then pull the associated country of origin from the second array. To make this easier, I am just pulling the entire line of the array, both IP and country.
I want to output this into a destination file.
This is what I've gotten so far. Am I doing this incorrectly?

#!/perl # $file1 = 'IPs.txt' ; # Name the file with IP addresses to be l +ooked up open(INFO, "<$file1" ) ; # Open the file @lines1 = <INFO> ; # Read it into an array close(INFO) ; # Close the file %file2 = 'Source.txt' ; # Name the file with source addresses open(INFO, "<%file2" ) ; # Open the file @lines2 = <INFO> ; # Read it into a two dimentional array close(INFO) ; # Close the file foreach $line1 (@lines1) # assign @lines1 to $line1, one at a time { foreach $line2 (@lines2) # assign @lines2 to $line2, one at a time { while $line1 != $line2 ; # Compare the entry from log gr +oup to source group { open(APPEND, ">>conversion.txt") ; print(APPEND "$line2" ; close(APPEND); } } }
Thank you
Carrie

updated by boo_radley : formatting, code tags.

Replies are listed 'Best First'.
Re: Comparing 1 list and 1 array
by kjherron (Pilgrim) on Jun 24, 2002 at 22:19 UTC
    Let me start with the most obvious issues:
    • The lines
      %file2 = 'Source.txt' ; open(INFO, "<%file2" ) ;
      is incorrect. You should use a '$' here, not a '%', since file2 is holding a simple string. For that matter, you could reuse $file1 from the previous block of code, or do without storing the filename in a variable altogether:
      open(INFO, '<Source.txt');
    • You're not verifying that any of the open() calls succeeds.
    • You're not removing end-of-line characters from any of the lines you're reading.
    • You're comparing strings containing only IP addresses (from the first file) to strings containing IP addresses and country names (from the second file). You're unlikely to find any matches.
    I would be inclined to use a hash to store the table of IP addresses and countries. You'd construct the hash something like this:
    my %table; open(IN, '<Source.txt') or die; while (<IN>) { chomp; my($ip, $country) = split(' ', $_, 2); $table{$ip} = $country; }
    You'll have to make sure that split() call correctly splits your input data into IP and country name, of course.

    Once you've done that--and assuming you've removed newlines from your input data--you can do country lookups like this:

    open(OUT, '>>conversion.txt') or die; foreach $line1 (@lines1) { if (exists $table{$line1}) { print OUT $line1, ' ', $table{$line1}, "\n"; } } close OUT;
      Thanks everyone for your help. I had wondered about using a %file, but I wanted a 2 dimentional array, so figured that would be how to get one. :/ I have taken out reading into $file, and opened the list straight into INFO, and gone with a hash table. My teammates asked that I print the IP if I couldn't find it in the table, so I added an else. I'm still working on the hash table, to make sure I don't need more than those 2 fields. Its been over 10 years since I did any Perl scripting, so I am WAY out of practice! Here is the second interation of code: #!/perl # open(INFO, '<IPs.txt' ) or die; @lines1 = <INFO> ; close(INFO) ; my %table open(IN, '<source.txt' ) or die; while (<IN>) { chomp; my($ip, $country) = split(' ',$_,2); $table($ip) = $country; } open(OUT, ">>conversion.txt") or die; foreach $line1 (@lines1) { if (exists $table($line1)) { if $line2 =~ /$line1/ ; print OUT $line1, ' ',$table($line1),"\n" ; else else print OUT $line1; } } close OUT;
Re: Comparing 1 list and 1 array
by bronto (Priest) on Jun 25, 2002 at 07:53 UTC

    Uhm... some things to point out here

    First, you don't use strict, and it hurts :-); it would have catched, for example:

    %file2 = 'Source.txt' ;

    which is an error, twice. First, because you are using an hash instead of a variable; second, because you are assigning an odd number of elements to an hash

    Second, you are slurping two files into memory, and that could be a pain if the files are too big, compared to your memory; probabily, you could do with reading one of the two line per line, and cache the other.

    Third, you have three cycles, put probabily you could just grep; more on this later, because there is a badder thing before:

    You open and close conversion.txt over and over: that is expensive! Why don't just open(APPEND, ">>conversion.txt") ; on top, close(APPEND); on bottom and print(APPEND "$line2") ; (watch the unbalanced parenthesis on that line!) when required?

    You are comparing $line1 and $line2 with !=: that's a numerical comparison. If you want to do the same test on strings, you should use ne instead.

    And now about the cycles: one way to compact them could be:

    foreach my $line1 (@lines1) { print APPEND grep $line1 ne $_,@lines2 ; }

    Take a look at the documentation (perldoc -f grep) to know more about grep

    Apart of the errors, keep studying perl. Maybe your program is not that good still, but it seems to me you are headed in the right direction ;-)

    Ciao!
    --bronto

Re: Comparing 1 list and 1 array
by kvale (Monsignor) on Jun 24, 2002 at 22:15 UTC
    One problem is with
    %file2 = 'Source.txt' ; # Name the file with source addresses open(INFO, "<%file2" ) ; # Open the file
    The `%file' indicates a hash, not what you want here. Change it to
    $file2 = 'Source.txt' ; # Name the file with source addresses open(INFO, "<$file2" ) ; # Open the file
    To extract the countries, you need to see if the IP address in $line1 is contained in $line2:
    open APPEND, ">>conversion.txt"; foreach $line1 (@lines1) { foreach $line2 (@lines2) { print APPEND $line2 if $line2 =~ /$line1/; } } close APPEND;
    That code is still just a start. In addition you will want to check whether the open statements succeed and you will want to use perl -w to turn on warnings that can catch typos like %file above.

    -Mark
      The code
      print APPEND $line2 if $line2 =~ /$line1/;
      is not a good way to look for an IP address in a line of text. First of all, it will match when you're searching for an IP that's a substring of the one contained in the text:
      $line1 = '1.2.3.4'; $line2 = '111.2.3.44'; print "match\n" if $line2 =~ /$line1/;
      Second, the periods in IP address strings will be interpreted as regular expression metacharacters:
      $line1 = '1.2.3.4'; $line2 = '11223344'; print "match\n" if $line2 =~ /$line1/;
Re: Comparing 1 list and 1 array
by caedes (Pilgrim) on Jun 25, 2002 at 00:59 UTC
    Why re-invent a buggy wheel when you can get one that works well for free? I've always used LookupIP, and it works great.

    -caedes

      If you aren't bound to your list of IP/countries, you can try with Geo::IP.

      BTW, practice makes perfect; I learned Perl this way, building small utilities.

      Happy working, Valerio