namishtiwari has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, Here is my piece of code,
#!/usr/bin/perl use strict; use warnings; my $Time_Stamp ; my $User_Name; my $Success; my $Failure; my $ErrorCode; my $ErrorMsg; my $logDir = $ARGV[0]; my $logPrefix = $ARGV[1]; my $log; my $line; die "usage: $0 <logDir> <logPrefix>" unless $logDir and $logPrefix; die "Log dir $logDir doesn't exist" unless -d "$logDir"; for my $logFile ( glob("$logDir/${logPrefix}*") ) { open($log, "<", $logFile) or die "Can't open $logFile for reading. +"; open(FP_OUT,">temp12") or die "cannot create file temp1 for writin +g"; print "Processing file $logFile...\n"; OUTER: while( $line = <$log> ) { chomp($line); if ($line =~ /^(.*)INFO:.*QNA Step - AUTH IN PROGRESS/) { $Time_Stamp = $1; printf FP_OUT "$Time_Stamp,"; QnA_search_for_sucess_or_failure() ; #QnA_search_for_userID() ; next OUTER; } } } sub QnA_search_for_sucess_or_failure { OUTER1: while ( $line = <$log> ) { if ($line =~ /QNA Auth.*Success\s*and\s*Complete/) { printf FP_OUT "Success,"; print STDOUT "In Success,\n"; $ErrorMsg="null"; while ( $line = <$log> ) { if ($line =~ /ArAuthFrameworkImpl::doPostAuth.*Au +thentication\s*mechanism\s*returned\s*\[(..*)\]\s*for\s*AuthIdentity\ +s*\[(..*)\]/) { print "$2\n"; printf FP_OUT "$2, ${1}, $ErrorMsg\n"; last OUTER1; } } } if($line =~ /Message.*QNA\s*Auth\s*Failed\((..*)\).*/) { $ErrorMsg=$1; printf FP_OUT "Failure,"; print STDOUT "In Failure,\n"; while ( $line = <$log> ) { if ($line =~ /ArAuthFrameworkImpl::doPostAuth +.*Authentication\s*mechanism\s*returned\s*\[(..*)\]\s*for\s*AuthIdent +ity\s*\[(..*)\]/) { print "$2\n"; printf FP_OUT "$2, ${1}, $ErrorMsg\n"; last OUTER1; } } } } } The below snippet is taken from the logfile.I am using while loops bec +ause there are many lines above and inbetween this snippet. <CODE> Tue May 19 22:55:13.649 2009 Morocco Standard Time INFO: pid 2172 t +id 3412: 160: 10083504: QNA Step - AUTH IN PROGRESS Tue May 19 22:55:13.649 2009 Morocco Standard Time INFO: pid 2172 t +id 3412: 160: 10083504: QNA Auth - Success and Complete, Returning SU +CCESS Tue May 19 22:55:13.665 2009 Morocco Standard Time INFO: pid 2172 t +id 3412: 17: 10083504: ArAuthFrameworkImpl::doPostAuth::1:10083487:: +Authentication mechanism returned [0] for AuthIdentity [12345] and for QnA failure the line comes like this in place of success line- +- Tue May 19 22:56:21.962 2009 Morocco Standard Time INFO: pid 2172 t +id 688: 160: 10083554: Err[115261735], Message: QNA Auth Failed(Inval +id Credentials), Repeating the challenge and Returning FAILED
when i ran the program like this--
C:\Perl Script>perl QnA_Authentication.pl . logfile.txt Processing file ./logfile.txt... In Success, 01503164 In Success, 01822755 In Failure, 01401058 Processing file ./logfile.txt.bak...
but in output file that is temp12, it is not printing anything. Kindly suggest wha is wrong here.

Replies are listed 'Best First'.
Re: wrong output
by GrandFather (Saint) on Jul 02, 2009 at 08:28 UTC

    After a little tidying to turn the code into a stand alone sample I don't see the problem you describe given the data you've supplied. Consider:

    use strict; use warnings; my $match = 'ArAuthFrameworkImpl::doPostAuth.*Authentication\s*mechanism\s*ret +urned\s*\[(..*­)\]\s*for\s*AuthIdentity\s*\[(..*)\]'; my $outText; open my $outFile, '>', \$outText; while (my $line = <DATA>) { chomp ($line); if ($line =~ /^(.*)INFO:.*QNA Step - AUTH IN PROGRESS/) { my $Time_Stamp = $1; printf $outFile "$Time_Stamp,"; QnA_search_for_sucess_or_failure (); } } close $outFile; print "\n\n$outText\n"; sub QnA_search_for_sucess_or_failure { while (my $line = <DATA>) { if ($line =~ /QNA Auth.*Success\s*and\s*Complete/) { printf $outFile "Success,"; print "In Success,\n"; my $ErrorMsg = "null"; while ($line = <DATA>) { if ($line =~ /$match/) { print "$2\n"; printf $outFile "$2, ${1}, $ErrorMsg\n"; return; } } } if (defined $line and $line =~ /Message.*QNA\s*Auth\s*Failed\((..*)\).*/) { my $ErrorMsg = $1; printf $outFile "Failure,"; print STDOUT "In Failure,\n"; while ($line = <DATA>) { if ($line =~ /$match/) { print "$2\n"; printf $outFile "$2, ${1}, $ErrorMsg\n"; return; } } } } } __DATA__ Tue May 19 22:55:13.649 2009 Morocco Standard Time INFO: pid 2172 t +id 3412: 160: 10083504: QNA Step - AUTH IN PROGRESS Tue May 19 22:55:13.649 2009 Morocco Standard Time INFO: pid 2172 t +id 3412: 160: 10083504: QNA Auth - Success and Complete, Returning SU +CCESS Tue May 19 22:55:13.665 2009 Morocco Standard Time INFO: pid 2172 t +id 3412: 17: 10083504: ArAuthFrameworkImpl::doPostAuth::1:10083487:: +Authentication mechanism returned [0] for AuthIdentity [12345] and for QnA failure the line comes like this in place of success line- +- Tue May 19 22:56:21.962 2009 Morocco Standard Time INFO: pid 2172 t +id 688: 160: 10083554: Err[115261735], Message: QNA Auth Failed(Inval +id Credentials), Repeating the challenge and Returning FAILED

    Prints:

    In Success, Tue May 19 22:55:13.649 2009 Morocco Standard Time ,Success,

    True laziness is hard work
      In the spirit of further simplification (and, just as important, encapsulating the output), this
      printf $outFile "Success,"; print "In Success,\n"; my $ErrorMsg = "null"; while ($line = <DATA>) { if ($line =~ /$match/) { print "$2\n"; printf $outFile "$2, ${1}, $ErrorMsg\n"; return; } }
      could be converted to a subroutine that would have parameters of "Success" and "null" (or "Error" and $1) passed to it.
      Hi, It did not display the whole output, as u can see some output is missing from there. It is not printing the $2 ie username and the errorcode. Regular expression looks ok to me and i tested also. Thanks NT
Re: wrong output
by jethro (Monsignor) on Jul 02, 2009 at 08:37 UTC

    When I ran your code with the snippet of the logfile you provided I got the following in temp12:

    Tue May 19 22:55:13.649 2009 Morocco Standard Time ,Success,12345, 0, +null

    which seems to be ok to me

      Yes that is the expected output according to the snippet, but for me it is not showing anything in the output file. I am puzzled what wrong i am doing. Thanks NT
      Hi, Actually my concern here is that the total output for QnA mismatches with the result of this code. I have a diffrent code that gives me the total count. I ran both the scripts on 6 logfiles, but for one the total no is 325 and for the above script is only 70, so there is big mismatch. If anyone can suggest anything regarding this by watching my code, then it will be great. Thanks NT

        The problem that you don't see anything in temp12 should not have anything to do with the perl program. Maybe you are looking into the wrong temp12 file. Check the directory listing. When you are writing to a file at a specific time the file must have that excact time in the directory listing. If not you are looking at a different file or a file you can't write to.

        About your concern: It was already suggested to you that your script will not find (multi line) log entries that start while a previous (mulit line) log entry isn't finished. Suggested remedies were to only look for the success and failure lines or keep a hash of entries so that you can work on more than one (multi line) log entry at the same time. Did you check if this is the case?

        To find out what is happening in your program, find the first log entry that should have been found but wasn't. Then copy the log file and edit the copy so that that specific log entry is at the start of the log file. If the log entry is found now, you know that previous log entries prevented this from being seen (quite probably the problem I mentioned above). If it still isn't found, use the perl debugger and single step through the code. Or use print statements to find out where the script diverts from your expectation

        So what convinces you that the "diffrent code that gives (you) the total count" is correct?

        It's hard to respond to such an assertion without knowing the "diffrent code" (sic).