Errors with sorting hashs

red-beard has asked for the wisdom of the Perl Monks concerning the following question:

I wrote a perl script to search my Sendmail logs for virus activety. This usually works fine but occasionally I get a "Use of uninitialized value in numeric comparison (<=>) at ./source_virus_count.pl line 84 (#1)". Line 84 happens to be my hashValueDescendingVirus subroutine. I'm suspecting it's pulling something from the logs but it's difficult to narrow down since my log file is 166,000 lines. Any ideas?


#!/usr/bin/perl
                                                                      
+                                                                     
+                  
# Populate a complex data structure with message id's, ip address's, a
+nd virus names. This is necesary since the ip address and virus name 
+are on separate lines but have the same message id.
# Count the number of times a virus was sent in descending order
# Count the number of times an IP address sent a virus in descending o
+rder
# Show each unique virus that was sent for each IP address.           
+                                                                     
+                                                                     
+        
                                                                      
+                                                                     
+                  
use warnings;
use diagnostics;
use strict;
                                                                      
+                                                                     
+                  
        my $email;
        my $ip_addr;
        my %ip_addr;
        my $virus;
        my %virus;
        my $Virus;
        my $sender;
        my $Sender;
        my $recipient;
        my $Recipient;
        my $message_id;
                                                                      
+                                                                     
+                  
print "Shows a count of each virus type:\n";
                                                                      
+                                                                     
+                  
open(FILE, "/var/log/maillog");
while(<FILE>) {
                                                                      
+                                                                     
+                  
if (/(?:\d|\D)+sendmail\[(?:\d)+\]:\s((?:\w)+):(?:\d|\D)+\[(\d+\.\d+\.
+\d+\.\d+)\]/) {
        $message_id = $1;
        $ip_addr = $2;
                unless ( $ip_addr eq "127.0.0.1" ) {
                        $email->{$message_id}{ip_addr} = $ip_addr;
                                                                      
+                                                                     
+                  
                }
        }
        elsif (/(?:\d|\D)+clamav-milter\[(?:\d)+\]:\s((?:\w)+):\sstrea
+m:\s(\d|\D+)\svirus from \<((?:\d|\D)+)\> to \<((?:\d|\D)+)\>/) {
        $message_id = $1;
        $virus = $2;
        $sender = $3;
        $recipient = $4;
                                                                      
+                                                                     
+                  
                        $email->{$message_id}{virus} = $virus;
                        $email->{$message_id}{sender} = $sender;
                        $email->{$message_id}{recipient} = $recipient;
                                                                      
+                                                                     
+                  
        }
}
close(FILE);
                                                                      
+                                                                     
+                  
foreach $message_id ( keys  %{ $email } ) {
                if ( $email->{$message_id}{virus} ) {
               $virus = $email->{$message_id}{virus};
                $ip_addr = $email->{$message_id}{ip_addr};
                $recipient = $email->{$message_id}{recipient};
                $sender = $email->{$message_id}{sender};
                #Counts total number of times a virus was sent
                $ip_addr{$virus}++;
                #Counts total number of virus's sent by IP address
                $virus{$ip_addr}++;
                #Counts total number of unique virus's per IP address
                $email->{$ip_addr}->{virus}{$virus}++;
                #Counts total number of unique senders per IP address
                $email->{$ip_addr}->{sender}{$sender}++;
                }
}
                                                                      
+                                                                     
+                  
sub hashValueDescendingVirus {
   $email->{$ip_addr}{virus}{$b} <=> $email->{$ip_addr}{virus}{$a};
}
                                                                      
+                                                                     
+                  
sub hashValueDescendingRecipient {
   $email->{$ip_addr}{recipient}{$b} <=> $email->{$ip_addr}{recipient}
+{$a};
}
                                                                      
+                                                                     
+                  
sub hashValueDescendingSender {
   $email->{$ip_addr}{sender}{$b} <=> $email->{$ip_addr}{sender}{$a};
}
                                                                      
+                                                                     
+                  
sub hashValueDescendingNum {
   $ip_addr{$b} <=> $ip_addr{$a};
}
                                                                      
+                                                                     
+                  
sub hashValueDescendingIp {
   $virus{$b} <=> $virus{$a};
}
                                                                      
+                                                                     
+                  
foreach $virus (sort hashValueDescendingNum (keys(%ip_addr))) {
                print "Count is $ip_addr{$virus} for $virus\n";
}
                                                                      
+                                                                     
+                  
print "\nShows uniques hosts with a virus count over 10:\n";
                                                                      
+                                                                     
+                  
foreach $ip_addr (sort hashValueDescendingIp (keys(%virus))) {
        if ($virus{"$ip_addr"} >= "10") {
                                                                      
+                                                                     
+                  
                print "\n$ip_addr sent the following virus's a total o
+f $virus{$ip_addr} times: \n";
                foreach $Virus (sort hashValueDescendingVirus (keys( %
+{ $email->{$ip_addr}{virus} } ))) {
                        print "$Virus was transmitted $email->{$ip_add
+r}{virus}{$Virus} times.\n";
                }
                print "\n";
                                                                      
+                                                                     
+                  
                if ($ip_addr eq "207.156.7.1") {
                        foreach $Sender (sort hashValueDescendingSende
+r (keys( %{ $email->{$ip_addr}{sender} } ))) {
                                print "Possibly spoofed address $Sende
+r was seen $email->{$ip_addr}{sender}{$Sender} times.\n";
                        }
                }
        }
}
[download]

Sample of logs: Aug 26 15:58:45 ns2b sendmail28597: i7QJwbXt028597: from=<rina@willmar.com>, size=17997, class=0, nrcpts=1, msgid=<x876126527.8746421781795333059@caelglobq>, proto=SMTP, daemon=MTA, relay=64-61-202-31.ips.cpinternet.com 64.61.202.31 Aug 26 15:58:45 ns2b clamav-milter11335: i7QJwbXt028597: stream: Worm.Zafi.B Intercepted virus from <rina@willmar.com> to <ostov@hillsboroughcounty.org>

Comment on Errors with sorting hashs Download Code

Replies are listed 'Best First'.
Re: Errors with sorting hashs by dragonchild (Archbishop) on Aug 27, 2004 at 13:33 UTC
To debug this, I would take the following steps: Have your script print out the line it's looking at Add `use warnings FATAL => qw( uninitialized numeric );` to the top of your script. Have it run. When it hits the naughty line in your log, it will print it out, then die, so the last line printed will be an example naughty-doer. ------ We are the carpenters and bricklayers of the Information Age. Then there are Damian modules.... sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon.* - flyingmoose I shouldn't have to say this, but any code, unless otherwise stated, is untested	[reply] [d/l]
Re^2: Errors with sorting hashs by red-beard (Scribe) on Aug 27, 2004 at 14:02 UTC
I'm getting the following when I do that. 65.207.129.252 sent the following virus's a total of 16 times: Use of uninitialized value in numeric comparison (<=>) at ./source_virus_count.pl line 70 (#1) (W uninitialized) An undefined value was used as if it were already defined. It was interpreted as a "" or a 0, but maybe it was a mistake. To suppress this warning assign a defined value to your variables. To help you figure out what was undefined, perl tells you what operation you used the undefined value in. Note, however, that perl optimizes your program and the operation displayed in the warning may not necessarily appear literally in your program. For example, "that $foo" is usually optimized into "that " . $foo, and the warning will refer to the concatenation (.) operator, even though there is no . in your program. Uncaught exception from user code: Use of uninitialized value in numeric comparison (<=>) at ./source_virus_count.pl line 70. main::hashValueDescendingVirus called at ./source_virus_count.pl line 99	[reply]
Re^3: Errors with sorting hashs by dragonchild (Archbishop) on Aug 27, 2004 at 14:15 UTC
Ok. You know the line that's causing a problem. Have you traced how that line moves through your program? Have you added print statements that tell you what's going on? What work have you done, given the new information? I'm not going to debug your program for you. I give advice. If you want work, I can give you my PayPal account to deposit $$ into. ------ We are the carpenters and bricklayers of the Information Age. Then there are Damian modules.... sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon.* - flyingmoose I shouldn't have to say this, but any code, unless otherwise stated, is untested	[reply]
Re^4: Errors with sorting hashs by red-beard (Scribe) on Aug 27, 2004 at 18:17 UTC
Re^3: Errors with sorting hashs by red-beard (Scribe) on Aug 27, 2004 at 14:23 UTC
I used Data::Dumper to show what was happening on lines 70 and 99. It looks like my subroutine is not getting the correct values. For example everything from the ip address till $VAR4 is correct. However my numeric comparison for $VAR4 is showing values from the $VAR2 comparison. 66.118.133.154 sent the following virus's a total of 19 times: $VAR1 = \{ 'sender' => { 'jdillon@karaco.com' => 1, '0000774243@nocmailsvc001.allthesites.org' = +> 1, 'se56092d.098@gwia.ci.tampa.fl.us' => 1, 'owner@zedo[2].txt' => 1, 'pobonline-feedback-138@lb.bcentral.com' => +1, 'al464718@aol.com' => 1, 'info@ebsengineering.com' => 1, 'abuse@gov.us' => 1, 'bids@enviro-logical.com' => 1, 'postmaster@ussurveyor.com' => 1, 'mmeyers@innet.com' => 1, 'rjnickson@netzero.net' => 1, 'owner@advertising[1].txt' => 1, 'fraysier@tds.net' => 1, 'dopicoproperties@msn.com' => 1, 'hussdrilling@earthlink.net' => 1, 'affiliatedsupply@hotmail.com' => 1, 's706moss@earthlink.net' => 1, 'arnolde@hillsboroughcounty.org' => 1 }, 'virus' => { 'Worm.SomeFool.P Intercepted' => 3, 'Worm.SomeFool.PIntercepted' => 16 } }; $VAR2 = \{ 'Worm.SomeFool.P Intercepted' => 3, 'Worm.SomeFool.PIntercepted' => 16 }; 65.207.129.252 sent the following virus's a total of 16 times: $VAR3 = \{ 'sender' => { 'c@0ie.ee' => 1, 'damian.sandoval@brooks.af.mil' => 1, 'ybev_@[ecxab.7' => 1, 'bp-es0es0cq.@n' => 1, 'e75@0.9' => 1, 'gb.f@5' => 1, 'ammer@langley.af.mil' => 1, 'bill_dougherty@urscorp.com' => 1, 'fl.@k.' => 1, '_c@ntriomin.gn-' => 1, '7k@7f9.c6' => 1, 'pdmeeks@access.mountain.net' => 1, '/ba-c@-da.id.je' => 1, '2/6@0.d' => 1, 'bleiter@huntconstructiongroup.com' => 1, 'e@h1.5' => 1 }, 'virus' => { 'Worm.SomeFool.ZIntercepted' => 7, 'Worm.SomeFool.Z Intercepted' => 9 } }; $VAR4 = \{ 'Worm.SomeFool.P Intercepted' => 3, 'Worm.SomeFool.PIntercepted' => 16 }; [download]	[reply] [d/l]
Re^4: Errors with sorting hashs by red-beard (Scribe) on Aug 27, 2004 at 14:31 UTC