Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I am writing a regex to find mutiple dates in a string if they exist, I have two formats and I dont know if there will one or two dates ahead of time When I run this I get the match for the first date NOT the second

The date formats can be:

Mar 11 08:02:08

Mar 11 08:02:08.32

Mar 11 2011 08:02:08

Mar 11 2011 08:02:08.32

$current_line = " Mar 11 08:02:08 172.28.17.253 Mar 11 2011 08:02:08 D +R-FW-1 :"; $current_line =~ m{ ( [A-Z]{1}[a-z]{2} #Month, 1 upper w\2 lower char [ ][ ]*? #one or more spaces [0-9][0-9]? #Day, one or more numbers [ ][ ]*? #one or more spaces [0-9]{2} #Hour, 2 digits [:] #delimeter, colon [0-9]{2} #Minute, 2 digits [:] #delimeter, colon [0-9]{2} #Seconds, 2 digits [.]? #decimal point if it exists [0-9]*? #Fracion of seconds if it exists )? #string may exist more than once .*? #Any text ( [A-Z]{1}[a-z]{2} #Month, 1 upper w\ 2lower char [ ][ ]*? #one or more spaces [0-9][0-9]? #Day, one or more numbers [ ][ ]*? #one or more spaces [0-9]{4} #Year, 4 digits [ ][ ]*? #one or more spaces [0-9]{2} #Hour, 2 digits [:] #delimeter, colon [0-9]{2} #Minute, 2 digits [:] #delimeter, colon [0-9]{2} #Seconds, 2 digits [.]? #decimal point if it exists [0-9]*? #Fraction of seconds if it exists )? #Date may exist more than once }gxms; if($1){ print "Date_1 = $1\n"}; if($2){ print "Date_2 = $2\n"};

Replies are listed 'Best First'.
Re: regex for multiple dates
by muba (Priest) on Jul 06, 2011 at 23:22 UTC

    First of all, that regex, despite it being commented, isn't very easy on the eyes. I know that this goes for regexen in general, but I find that \s+ is much cleaner than [ ][ ]*?. But I'm just nitpicking.

    The last comment in your code confuses me a bit - "Date may exist more than once", however ? means "0 times or once". Then, later on, you rely on $1 and $2 to read out the matched dates, suggesting that you only care about the first two occurences of a date substring.

    What I also find confusing is that you say that dates can optionally specify a year, but your first regex doesn't allow for those for digits. Your regex consists of two halves, both intended to match a date that should be composed according to a specific format:

    Month + (space) + Day + (space) + (Optional: Year + (space) ) + Hour + (colon) + Minute + (colon) + Seconds + (Optional: (decimal point) + Fraction )
    but the first half of your regex never tries to match the year. In other words, the two attempts in your regex to capture strings that should adhere to the same format aren't compatible with each other.

    So I've taken the liberty to rewrite your regex -and your code in general-, so that it:
    1) accepts all of the formats you've specified for a date stering;
    2) captures all date substrings in the string, not just the first two 3) allows you to work with only the first two dates anyway, if that's what you want

    use strict; use warnings; my $current_line = " Mar 11 08:02:08 172.28.17.253 Mar 11 2011 08:02:0 +8 DR-FW-1 :"; $current_line .= "And also Apr 1 11:12:13 -- April's Fool is a fun day + :)"; my @dates = $current_line =~ m/ ( # Match and capture... (?:Jan|Feb|Mar| # ... one of the twelve months Apr|May|Jun| # I prefer to be explicit about this: +it's Jul|Aug|Sep| # a very limited set of strings that w +e accept Oct|Nov|Dec # and [A-Z][a-z]{2} is too unrestricti +ve ) \s+ # ... and one or more spaces \d\d? # ... and one or two digits for the day, b +ut note # that this will also match for Feb 30 +, # which doesn't exist, or for # a day such as 54 \s+ # ... and one or more spaces (?:\d{4}\s+)? # ... and optionally four digits for the y +ear, # followed by one or more spaces \d\d:\d\d:\d\d # ... HH:MM:SS, but note that this will al +so # acccept hours such as 34, or minutes + such as # 84, so this isn't the best we can do +! (?:\.\d*)? # ... and optionally a decimal point, whic +h, if # present, is optionally followed by f +raction # of second ) # End of capture. /gxms; print "I got ", scalar(@dates), " dates:\n"; print " => $_\n" for @dates; print "But the first two are $dates[0] and $dates[1]\n";

    Output:

    I got 3 dates: => Mar 11 08:02:08 => Mar 11 2011 08:02:08 => Apr 1 11:12:13 But the first two are Mar 11 08:02:08 and Mar 11 2011 08:02:08

Re: regex for multiple dates
by AnomalousMonk (Archbishop) on Jul 07, 2011 at 01:51 UTC

    This is more heavily into date parsing and I think there are probably modules better suited to this (e.g., Date::Parse?), but here's an example of a structured approach using a nifty regex feature from 5.10+ (caution: this code is obviously not exhaustively tested):

    >perl -wMstrict -le "my @dates = ( 'Mar 11 08:02:08', '11 Mar 08:02:08', 'Mar 11 08:02:08.32', '11 Mar 08:02:08.32', 'Mar 11 2011 08:02:08', '2011 Nov 11 08:02:08', 'Mar 11 2011 08:02:08.32', '11 Dec 2011 08:02:08', '--------------------', 'Mar 32 2011 08:02:08', '2011 08:02:08', 'Mar 11 2011 .00', ); ;; my @months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec); my $mname = qr{ (?<MON> @{[ join '|', @months ]}) }xms; my $mday = qr{ (?<DAY> 0 [1-9] | [12] \d | 3 [01]) }xms; my $day = qr{ $mname \s+ $mday | $mday \s+ $mname }xms; my $year = qr{ (?<YEAR> (?: 19 | 20) \d\d) }xms; my $date = qr{ $day (?: \s+ $year)? | (?: $year \s+)? $day }xms; my $hms = qr{ (?<HMS> \d\d (?: : \d\d){2}) }xms; my $hund = qr{ (?<HUND> \. \d{2}) }xms; my $time = qr{ $hms $hund? }xms; ;; DATE: for my $d (@dates) { my $parsed = $d =~ m{ \A $date \s+ $time \z }xms; if (not $parsed) { warn qq{bad date: '$d'}; next DATE; } my $day = $+{DAY}; my $mon = $+{MON}; my $yr = $+{YEAR} || '1999'; my $t = $+{HMS}; my $h = $+{HUND} || '.00'; my $canonical_date = qq{$mon $day $yr $t$h}; printf qq{%-25s -> '%s' \n}, qq{'$d'}, $canonical_date; } " 'Mar 11 08:02:08' -> 'Mar 11 1999 08:02:08.00' '11 Mar 08:02:08' -> 'Mar 11 1999 08:02:08.00' 'Mar 11 08:02:08.32' -> 'Mar 11 1999 08:02:08.32' '11 Mar 08:02:08.32' -> 'Mar 11 1999 08:02:08.32' 'Mar 11 2011 08:02:08' -> 'Mar 11 2011 08:02:08.00' '2011 Nov 11 08:02:08' -> 'Nov 11 2011 08:02:08.00' 'Mar 11 2011 08:02:08.32' -> 'Mar 11 2011 08:02:08.32' '11 Dec 2011 08:02:08' -> 'Dec 11 2011 08:02:08.00' bad date: '--------------------' at -e line 1. bad date: 'Mar 32 2011 08:02:08' at -e line 1. bad date: '2011 08:02:08' at -e line 1. bad date: 'Mar 11 2011 .00' at -e line 1.

    Update: The central parsing regex above was originally
        m{ \A $date \s+ $time $hund? \z }xms
    but should have been and is now
        m{ \A $date \s+ $time \z }xms
    The  $hund? was completely redundant and this fix produces no change in the output.

Re: regex for multiple dates
by Somni (Friar) on Jul 06, 2011 at 23:43 UTC
    Can't say I dig your regex all that much. Too many uses of unnecessary brackets, and an avoidance of more readable character classes. It's good you started with /x, but there is such thing as too much vertical whitespace.

    There are a few ways of accomplishing your task, depending on what you want from the result. There's yours, where everything is jammed into one regex; this is ideal if you don't need anything beyond that (say, to capture the individual pieces, or easily associate something with which pattern matched). For just pulling dates out, it typically works.

    However, when matching multiple possible patterns I tend to go with an incremental parser. You define all of your patterns, and organize them such that the more exact match first. For example, with your data set:

    my $month_abv = qr/[[:upper:]][[:lower:]]{2}/; my @date_patterns = ( # Mar 11 2011 08:02:08.32 ['Mmm DD YYYY HH:MM:SS.uu', qr/$month_abv \s+ \d\d? \s+ \d{4} \s+ \d\d:\d\d:\d\d\.\d\d /x], # Mar 11 2011 08:02:08 ['Mmm DD YYYY HH:MM:SS', qr/$month_abv \s+ \d\d? \s+ \d{4} \s+ \d\d:\d\d:\d\d /x], # Mar 11 08:02:08.32 ['Mmm DD HH:MM:SS.uu', qr/$month_abv \s+ \d\d? \s+ \d\d:\d\d:\d\d\.\d\d /x], # Mar 11 08:02:08 ['Mmm DD HH:MM:SS', qr/$month_abv \s+ \d\d? \s+ \d\d:\d\d:\d\d /x], ); my $line = q{ Mar 11 08:02:08 172.28.17.253 Mar 11 2011 08:02:08 D+R-FW-1 : Jan 1 2011 00:00:00.07; }; DATE: until ($line =~ /\G\z/gc) { foreach my $pat (@date_patterns) { my($form, $re) = @$pat; if ($line =~ /\G\b($re)\b/gc) { say "Matching date form '$form': $1"; next DATE; } } $line =~ /./sgc; # no matches, bumpalong }

    Each of your possible patterns are defined (in length order, so that the smaller patterns are tried last). You get free metadata with it; in this case, a simple string that describes the pattern. In more complicated cases you'd want a function that reformats the date, or shoves it into an object, etc. based on captures within the pattern.

    Incidentally, while I did change your [A-Z] and [a-z] to something a little more locale friendly ([[:upper:]] and [[:lower:]]), what the month pattern really should be is simply \w+, or perhaps [[:alpha:]]. For your data set it probably doesn't matter, but not all languages provide three characters for the month, or provide it with the first letter uppercased. Some locales may not even use all letters for month names. The rest of the pattern in each case should filter out false positives.

    In addition to changing the metadata, you could change the \b delimiters; perhaps your dates are all before and after whitespace, or some punctuation. Making it specific to your data set can eliminate more false positives.

    In order to accomplish this with one big regex you would simply join the regexes in @date_patterns with |, and commence matching as normal.

      Thanks to both of you, I know my code was hard to read. I'm new regex and having a very hard time understanding the syntax and writing the code so that I can go back and understand what I wrote.

      The text I get has one or two dates in it and the format is different in it may not have the year, I didnt know how to make a portion of the check optional.

      By looking at the code you guys wrote I think I see what I'm doing wrong, I'm rewriting the code. thanks for the help

        I think it's often helpful to start with zero generality. Figure out the tricky bits of the match operation and regular expression pattern first, then generalize the pattern.

        Let's assume you want to match one or more of these specific timestamps in a string:

            Mar 11 08:02:08
            Mar 11 08:02:08.32
            Mar 11 2011 08:02:08
            Mar 11 2011 08:02:08.32
        

        This expression will match and capture them:

        m/(Mar 11 (?:2011 )?08:02:08(?:\.32)?)/g;

        Simple.

        Now it's easy to generalize this pattern as much as you need to for your specific application. The following generalization is probably sufficient:

        m{((?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s+ \d\d?\s+(?:(?:19|20)\d\d\s+)?\d\d:\d\d:\d\d(?:\.\d\d)?)}gx;

        Jim

Re: regex for multiple dates
by locked_user sundialsvc4 (Abbot) on Jul 07, 2011 at 02:50 UTC

    I suggest you read perldoc perlre and find the spot about “global matching,” the i and g modifiers; and the pos function (perldoc -f pos).

    You can, in short, ask Perl to look for a pattern and to then remember where it left off, so that you can search for other patterns in the rest of the string.   pos also allows you to go back to those positions.

    (Also note that undef, not zero, is the correct thing to reset pos to, when you want to start searching from the start of the string again!   Yeah, you’re darned right “I speak from experience” on that one ...)