Real global substitution, what's the best way?

mirod has asked for the wisdom of the Perl Monks concerning the following question:

This bugged me for a while so I thought I'd mention the solution I found, as well as ask if there is a better way to do it:

Basically I wanted to do a rather complex substitution globally but /g did not work for me. The reason /g did not work is that I need part of the string that has already be used in order to do each match.

I guess a simple example will illustrate the problem better: in the string 'a1b1a2b1b2a3b1b2b3'db1b2 I want to move all the b\d+ following a\d+ to the end of the string and wrap them in (). In this case the simple s{(a\d+)(b\d+)(.*)}{$1$3($2)}g will not work: it will only move the first b, as the regexp engine has to go all the way to the end of the string to match the (.*) part.

There might be ways to do this in one substitution, and I'd like to hear about them, but in fact the real regexps are a bit more complex, so easy tricks are not likely to work, I have searched for them for a while ;--( The main problems are that the (a\d+) is of variable length, which prevents using a look behind (?<=a\d+), each (b\d+) has to be processed separately, and the final (.*) will go through the whole string anyway...

In any case here is the solution I found: use a while with an empty body: while( $string=~ s{(a\d+)(b\d+)(.*)}{$1$3($2)}){}. This works fine, is conceptually clean, but I don't like the empty {}, it looks just weird. Which makes me wonder if there is an other way to do this, one that would be blessed by better hackers than me?

Here is a toy script if you want to play with the problem:

#!/bin/perl -w
use strict;

while( my $string=<DATA>)
  { # this works
    while( $string=~ s{(a\d+)(b\d+)(.*)}{$1$3($2)}){}
    # this does not
    #$string=~ s{(?<=a\d+)(b\d+)(.*)}{$3($2)}g;

    print "string: $string\n"; 
  }


__DATA__
a1b1b2b3
a1b1a12b1b2a3b1b2b3db1b2
[download]

Comment on Real global substitution, what's the best way? Download Code

Replies are listed 'Best First'.
Re: Real global substitution, what's the best way? by blakem (Monsignor) on Dec 08, 2001 at 02:11 UTC
For your case I think `1 while s///` is as good as its going to get. Howver, if you wanted to match patterns that overlap the following code is fun to play with: `#!/usr/bin/perl -w use strict; $_ = '1ab2ab3ab4'; # ^^^^ # ^^^^ # ^^^^ # three substrings match /\dab\d/ my @nonoverlap = /\dab\d/g; # this will only find two of them my @overlap; while (/(\dab\d)/g) { # this will find all three push(@overlap,$1); pos($_) = $-[0] + 1; } print "nonoverlap: @nonoverlap\n"; # 1ab2 3ab4 print "overlap: @overlap\n"; # 1ab2 2ab3 3ab4` [download] The pos() assignment trick to find overlapping matches comes straight of out chapter 6 of japhys book which is probably the best /g documentation out there. -Blake	[reply] [d/l] [select]
Re: Real global substitution, what's the best way? by dws (Chancellor) on Dec 08, 2001 at 00:42 UTC
Use `((:?b\d+)+)` to match a sequence. The `(b\d+)` you have will only match a single occurance.	[reply] [d/l] [select]
Re: Re: Real global substitution, what's the best way? by mirod (Canon) on Dec 08, 2001 at 00:50 UTC
I can only match one occurence at a time, because I need to process each of them separately : want to wrap each one, separately, in `()`, so matching them all would force me to then include a some processing on the right side of the substitution, which is possible, but, I think, not as easy to write as what I have right now. It could be slightly faster though, although it is not really important in my software. I admit it the example looks quite contrived but it really makes sense in the real code!	[reply]
(tye)Re: Real global substitution, what's the best way? by tye (Sage) on Dec 08, 2001 at 12:32 UTC
I came up with this: `my $beg= ""; # Beginning of new string my $end= ""; # Ending of new string my $prev= ""; # Previous token my $str= "'a1b1a2b1b2a3b1b2b3'db1b2"; for my $sub ( split /([ab]\d+)/, $str ) { next if "" eq $sub; if( $prev =~ /^a/ && $sub =~ /^b/ ) { $end .= "($sub)"; } else { $beg .= $sub; } $prev= $sub; } print "$beg$end\n";` [download] - tye (but my friends call me "Tye")	[reply] [d/l]
Re: Real global substitution, what's the best way? by dragonchild (Archbishop) on Dec 08, 2001 at 00:40 UTC
While not aesthetically beautiful, it makes perfect sense to do it that way. Now, of course, japhy will come up with a hideously complex regex using backwards-interpolation-badnesses, but that's japhy. :-) ------ We are the carpenters and bricklayers of the Information Age. Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.	[reply]
Re: Real global substitution, what's the best way? by traveler (Parson) on Dec 08, 2001 at 01:21 UTC
In C I am used to: `while (expr) ;` [download] I don't like the `while (expr) {}` in perl. Instead I personally prefer `1 while (expr);` Maybe I am just strange. HTH, --traveler	[reply] [d/l] [select]