ExReg has asked for the wisdom of the Perl Monks concerning the following question:

I have run into a little problem in extracting information from a bunch of files. As I loop through all the files, I slurp each one in and then look for information matching certain criteria. For simplicity's sake, lets say that the file contents is

$fc = 'abcdfoofrobnicatebardefforspambazghi';

I then search for certain patterns in the file and capture them using a regex, say:

$re1 = qr/fo.*?ba./;

I capture them into an array for later analysis thus:

@excerpts = $fc =~ /$re1/g;

The above yields two entries in @excerpts: 

foofrobnicatebar
forspambaz

[download]

Later, when I get to analyzing the excerpts, I want to search each one for patterns within each match using almost the same regex with capture groups:

$re2 = qr/(fo.)(.*?)(ba.)/;

This gives me 

foo frobnicate bar
for spam baz

[download]

My question is this: The expressions I am really using are much more complicated in the two regexes, $re1 and $re2, and there are orders of magnitude more captures than the two shown here. If I make a change to one regex, I have to change the other. This can be a hassle in big ugly expressions, and I have already failed once to notice they were not in sync. It would be nice to have only one regex, but if I were to use the second expression in the first capture, I would get 

@excerpts = $fc =~ /$re2/g;

foo frobnicate bar for spam baz

[download]

Six entries in @excerpts

Having capture groups in a /g capture makes each capture group add an element to the array instead of the whole regex adding one. Is there an easier way than having two almost identical regexes? Can I have one regex with capture groups, but keep it from making each capture group create an array element under /g?

Replies are listed 'Best First'.
Re: Behavior of /g when there are capture groups
by Eily (Monsignor) on Apr 21, 2016 at 14:57 UTC

    If you call the regex in scalar context (ex: as a conditional) /g will make it possible to work one match at a time, allowing you to have finer control over what you do with the captures. 

    #use Data::Dumper;
my %res;
while (/(fo.)(.*?)(ba.)/g)
{
  $res{$&} = [ @-[1..3] ];
}
print Dumper \%res;

    [download]

    Edit: the slice should be on indices 1..3, not 0..2.

[reply]
[d/l]

      This could be generalized to extract substrings, not just substring initial offsets; also to avoid  $& which just slows everything down. (The many evals might slow things down, though. Oh, well...) (Also tested under 5.8.9.) 

      c:\@Work\Perl>perl -wMstrict -le
"use Data::Dumper;
 ;;
 $_ = 'abcdfoofrobnicatebardefforspambazghi';
 ;;
 my $re = qr{ (fo.) (.*?) (ba.) }xms;
 ;;
 my %res;
 while (/($re)/g) {
   $res{$1} = [ map eval qq{\$$_}, 2 .. $#- ];
   }
 print Dumper \%res;
"
$VAR1 = {
          'forspambaz' => [
                            'for',
                            'spam',
                            'baz'
                          ],
          'foofrobnicatebar' => [
                                  'foo',
                                  'frobnicate',
                                  'bar'
                                ]
        };

      [download]

      Give a man a fish:  <%-{-{-{-<

[reply]
[d/l]
[select]

        I should have tested my code, I got confused and thought @- held the submatches themselves, not the indexes. So I actually meant $res{$&} = [ $1, $2, $3]; but got lazy :). And $res{$1} = [ $2, $3, $4 ] works without $& with the extra parentheses.

        Your proposition has the benefit of not requiring to know the inner regex and its number of captures :)

[reply]
[d/l]
[select]

      Thanks! I was unaware of that trick.

[reply]
Re: Behavior of /g when there are capture groups (join)
by tye (Sage) on Apr 21, 2016 at 14:54 UTC 
    my @groups = ( 'fo.', '.­*?', 'ba.' );
my $find = join '', @groups;
my $capture = join '', map "($_)", @groups;
my $fc = read_my_file();
my @excerpts = $fc =~ /$find/g;
my @parts = $excerpts[0] =~ /$capture/;

    [download]

    [Updated: Changed [ ... ] to ( ... ) on the first line -- thanks for the private correction.]

    - tye        

[reply]
[d/l]
[select]

      Nice succinct way of constructing the two regexes from one source. I think that may do the trick.

[reply]
Re: Behavior of /g when there are capture groups (/n)
by tye (Sage) on Apr 21, 2016 at 14:57 UTC 
    my @excerpts = $fc =~ /(?n:$re2)/g;

    [download]

    From perlre:

    n
    Prevent the grouping metacharacters () from capturing. This modifier, new in 5.22 ...

    - tye        

[reply]
[d/l]

      That is exactly what I was looking for. Too bad I only have 5.8.8...

[reply]
[reply]
[d/l]
Re: Behavior of /g when there are capture groups
by AnomalousMonk (Archbishop) on Apr 21, 2016 at 18:30 UTC

    There're only two ways I know to avoid "spurious" capture groups in Perl 5.8.x regexes: don't use capture groups at all, or grep them away. Here's another approach that might or might not fit with your "... expressions [that] are much more complicated ..." in the pairs of regexes (update: tested under ActiveState 5.8.9): 

    c:\@Work\Perl>perl -wMstrict -MData::Dump -le
"print qq{Perl version: $] \n};
 ;;
 my $fc =
  'abcdfoofrobnicatebardefforspambazghixyproblemaaaxyzzylettuce';
 ;;
 my $min = qr{ .*? }xms;
 ;;
 my @groups = (
   [ qr{ fo. }xms, $min, qr{ ba. }xms, ],
   [ qr{ xy. }xms, $min, qr{ le. }xms, ],
   );
 ;;
 my ($find) =
   map  qr{ $_ }xms,
   join q{ | },
   map  qr{ @$_ }xms,
   @groups
   ;
 print qq{$find \n};
 ;;
 my ($capture) =
   map  qr{ $_ }xms,
   join q{ | },
   map  { map qr{ $_ }xms, join ' ', map qr{ ($_) }xms, @$_ }
   @groups
   ;
 print qq{$capture \n};
 ;;
 my @excerpts = $fc =~ m{ $find }xmsg;
 dd \@excerpts;
 ;;
 for my $exrpt (@excerpts) {
   my @parts = grep defined, $exrpt =~ m{ $capture }xmsg;
   printf qq{'$_' } for @parts;
   print '';
   }
"
Perl version: 5.008009

(?msx-i: (?msx-i: (?msx-i: fo. ) (?msx-i: .*? ) (?msx-i: ba. ) ) | (?m
+sx-i: (?msx-i: xy. ) (?msx-i: .*? ) (?msx-i: le. ) ) )

(?msx-i: (?msx-i: (?msx-i: ((?msx-i: fo. )) ) (?msx-i: ((?msx-i: .*? )
+) ) (?msx-i: ((?msx-i: ba. )) ) ) | (?msx-i: (?msx-i: ((?msx-i: xy. )
+) ) (?msx-i: ((?msx-i: .*? )) ) (?msx-i: ((?msx-i: le. )) ) ) )

["foofrobnicatebar", "forspambaz", "xyproblem", "xyzzylet"]

'foo' 'frobnicate' 'bar'
'for' 'spam' 'baz'
'xyp' 'rob' 'lem'
'xyz' 'zy' 'let'

    [download]
    (Please forgive the wrap-around in the long regex print statements. I've tried to mitigate the effects; I hope I didn't introduce any | too much cruft.)

    Update: Of course, this is an attempt to generalize tye's approach above to deal with unlimited pairs of regexes. Whether it applies to your situation is a separate question.

    Give a man a fish:  <%-{-{-{-<

[reply]
[d/l]
[select]
Re: Behavior of /g when there are capture groups
by Anonymous Monk on Apr 21, 2016 at 15:50 UTC 
    @excerpts = ();
push @excerpts, $1 while $fc =~ /($re2)/g;

    [download]
[reply]
[d/l]

      Awesome!

[reply]

Back to Seekers of Perl Wisdom