Scan Files - Match String

deelinux has asked for the wisdom of the Perl Monks concerning the following question:

Hi I want to search some logs and have the created the below, but match fails. (newish to scripting) The text is similar to the below, and I want to match on current date (no time) and then on ERROR or WARN Example text 2017-01-04 08:14:19,753 - 08:14:19,752 WARN 2017-01-04 08:14:19,753 - 08:14:19,752 ERROR Its not matching, but does if I use just the date, What I want is the date anything in between and then on the ERROR or WARN, if anyone can shed some light that would be good or an easier way. Thanks Dee

 
MyVariables
$MYFOLDER = "/tmp/logs"
#Severity Levels
$SLEVEL1 = "ERROR";
$SLEVEL2 = "WARN";

#Mycustom Date - this gives  2017-01-04
$datestring = strftime "%Y-%m-%d", localtime; 


#MyMatches Pattern
#$match1 = "^$datestring\s+\d{2}:\d{2}:\d{2},\d{3}\s+-\s+\d{2}:\d{2}:\
+d{2},\d{3}\s+$SLEVEL1";


sub MyScanLogs {
  @files = `find $MYFOLDER -name "*.log"`;
    foreach $file(@files) {
     open FILE, "$file" or die "Unable to open files";
        while(<FILE>) {
          
            if ($_ =~ /$match1/) {
                print "$file $_\n";
               
                 }
                     }
           }
    }


MyScanLogs
[download]

Comment on Scan Files - Match String Download Code

Replies are listed 'Best First'.
Re: Scan Files - Match String by haukex (Archbishop) on Jan 04, 2017 at 18:31 UTC
Hi deelinux, The reason your code isn't working is the way you've saved your regex. In strings surrounded by double quotes, backslash escapes like `\d` and `\s` mean something different than in regexes. In fact, `\d` and `\s` are not valid in double-quoted strings, only regexes (for details, see Quote Like Operators, perlrebackslash and Gory details of parsing quoted constructs). You can see this if you use Data::Dumper to look at the variable (see also the Basic debugging checklist): `my $match1 = "^$datestring\s+\d{2}:\d{2}:\d{2},\d{3}\s+-\s+\d{2}:\d{2} +:\d{2},\d{3}\s+$SLEVEL1"; use Data::Dumper; $Data::Dumper::Useqq=1; print Dumper($match1); __END__ $VAR1 = "^2017-01-04s+d{2}:d{2}:d{2},d{3}s+-s+d{2}:d{2}:d{2},d{3}s+ERR +OR";` [download] Also, if you had turned on warnings, you would have gotten warnings about this, which is why in my other post I recommended Use strict and warnings! Personally, I think the best way to fix this is to use `qr//`. If I do the following, your code starts working: `my $match1 = qr/^$datestring\s+\d{2}:\d{2}:\d{2},\d{3}\s+-\s+\d{2}:\d{ +2}:\d{2},\d{3}\s+$SLEVEL1/;` [download] Hope this helps, -- Hauke D	[reply] [d/l] [select]
Re^2: Scan Files - Match String by deelinux (Novice) on Jan 04, 2017 at 18:47 UTC
Hello Hauke and everyone else that commented, many thanks all. I have taken note about the correct way to submit a questions and how to do regex the right way. In the end I really liked the easier method by Hauke, `if (/\Q$datestring/ && /ERROR\|WARN/)` This worked a treat, so thanks again, I'll play around with the Regex way and see if I can get that working. :-)	[reply] [d/l]
Re: Scan Files - Match String (updated) by haukex (Archbishop) on Jan 04, 2017 at 18:12 UTC
Hi deelinux, Please use `<code>` tags for the sample input as well. Also, your code does not compile, I had to add `use POSIX 'strftime';` and a missing semicolon. Its not matching When I run your code against your sample input, it appears to work for me, the output I get is: `/tmp/logs/test.log 2017-01-04 08:14:19,753 - 08:14:19,752 WARN /tmp/logs/test.log 2017-01-04 08:14:19,753 - 08:14:19,752 ERROR` [download] Is this not what you were expecting? Update 2: Never mind, I missed the fact that `$match1` was commented out (due to the missing `use strict;`). The advice below still applies! Please see How do I post a question effectively? (Update: And post a Short, Self-Contained, Correct Example. Code that complies cleanly and some more sample input and its expected output would be good, since I'm unclear on "What I want is the date anything in between and then on the ERROR or WARN") if anyone can shed some light that would be good or an easier way In general, you should Use strict and warnings, it helps prevent mistakes. Second, you don't need to make your regexes so complex, and you can split them into several regexes if it makes it easier to think about. For example, to match on the current date and then on "ERROR" or "WARN", you could just write `if (/\Q$strftime/ && /ERROR\|WARN/) ...` Update 3: See also my other reply for a fix to your problem. Hope this helps, -- Hauke D	[reply] [d/l] [select]
Re: Scan Files - Match String by neilwatson (Priest) on Jan 04, 2017 at 18:02 UTC
Start with a small simple regex and add to if after each success. Then you'll know which part is failing. Neil Watson watson-wilson.ca	[reply]