Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to work out how Perl substitutions work. We're told that backreference patterns can be used in the substitution by specifying \1, \2 (etc), but it doesn't seem to work.

Snippet below. It tries to turn the string 'Hello sailor' into 'Goodbye sailor', but I get 'Goodbye \1' instead.

(Obviously this isn't sensible way to convert one string to the other, but this is just an example.)

#!/usr/bin/perl
use strict;
use diagnostics;
use warnings;

my $string = 'Hello sailor';
my $regex = 'Hello (.*)';
my $substitution = 'Goodbye \1';

$string =~ s/$regex/$substitution/;
print "string is now $string \n";

# I wanted:     string is now Good bye sailor
# I got:          string is now Good bye \1

[download]

Replies are listed 'Best First'.
Re: Substitution backreference woes
by Laurent_R (Canon) on Jan 24, 2015 at 15:14 UTC
    \1 is a back reference within the same regular expression which may be useful, among other things, to look for a repeated word. For example: 
      DB<1> $c = "The black dog and the black cat";

  DB<2> print "Matched" if $c =~ /(bl..k).+\1/;
Matched

    [download]
    In a substitution expression, you should rather use $1.

    Je suis Charlie.
[reply]
[d/l]
Re: Substitution backreference woes
by AnomalousMonk (Archbishop) on Jan 24, 2015 at 21:36 UTC

    It's important to remember is that the replacement portion of a  s/search/replace/ substitution (the part between the second  // pair) exactly obeys the single- or double-quote interpolation rules of any single- or double-quoted literal. The substitution above could be re-written as
        s{search}"replace"
    or, if no interpolation was wanted, as
        s{search}'replace'
    instead.

    String interpolation in Perl does not nest. (A Perl string is not an AST macro.) If I start with a string
        my $s = 'fee \n fie $foo fum';
    (again, single-quoting completely defeats interpolation), I get a string that is literally
        fee \n fie $foo fum
    If I then intepolate this string into another string, the second string is not re-scanned after interpolation to handle anything that looks like it might possibly be further interpolated. 

    c:\@Work\Perl>perl -wMstrict -le
"my $s = 'fee \n fie $foo fum';
 print qq{'$s'};
 ;;
 my $t = qq{zit $s zot};
 print qq{'$t'};
"
'fee \n fie $foo fum'
'zit fee \n fie $foo fum zot'

    [download]

    You may be looking for some sort of text templating system, e.g., Text::Template. The  s///ee 'trick' shown by LanX above works, but lacks generality.

    Give a man a fish:  <%-(-(-(-<

[reply]
[d/l]
[select]
Re: Substitution backreference woes
by 2teez (Vicar) on Jan 24, 2015 at 13:21 UTC

    With my $substitution = 'Goodbye \1'; which is obviously not a regex, which regex "group" are you back-referring to?

    Check this however: 

    my $string = "Hello sailor";

my $regex  = qr/Hello (.*)/;

$string =~ s/$regex/Goodbye \1/;

print "string is now $string \n";

    [download]

    If you tell me, I'll forget.
    If you show me, I'll remember.
    if you involve me, I'll understand.
    --- Author unknown to me
[reply]
[d/l]
[select]
Re: Substitution backreference woes
by Anonymous Monk on Jan 24, 2015 at 12:51 UTC
[reply]

      Forgive me, but the working examples do not seem to cover the use of \1 (etc) in the replacement expression.

[reply]
        Forgive me, but the working examples do not seem to cover the use of \1 (etc) in the replacement expression.
        It's covered in perlop
        Unlike sed, we use the \<digit> form in only the left hand side. Anywhere else it's $<digit>.
        Perl warns about it, too: 
        $ perl -wE '$_ = "foo"; s/(foo)/bar \1/; say'
\1 better written as $1 at -e line 1.
bar foo

        [download]
        We're told that backreference patterns can be used in the substitution by specifying \1, \2 (etc), but it doesn't seem to work.
        Who told you that?
[reply]
[d/l]
Re: Substitution backreference woes
by LanX (Saint) on Jan 24, 2015 at 13:06 UTC
    You are escaping 1 in your code, but what you need is backslash 1, so try \\1 in your variable.

    You need to realize that escaping in normal strings is different from "meta characters" in regular expressions.

    (yet untested)

    update

    wasn't the problem here.

    Cheers Rolf

    PS: Je suis Charlie!

[reply]
[d/l]

      That seems to produce the same result, namely:

      string is now Goodbye \1

      #!/usr/bin/perl
use strict;
use diagnostics;
use warnings;

my $string = 'Hello sailor';
my $regex = 'Hello (.*)';
my $substitution = 'Goodbye \\1';

$string =~ s/$regex/$substitution/;
print "string is now $string \n";

# I wanted:     string is now Goodbye sailor
# I got:          string is now Goodbye \1

      [download]
[reply]
[d/l]
        yeah sorry I wasn't able to test from Android, the problem is that the substitution part of s/// is supposed to be a literal. Otherwise you have to apply the /e eval modifiers.

        my $string = 'Hello sailor';
my $regex = 'Hello (.*)';
# my $substitution = 'Goodbye \1';

$string =~ s/$regex/Goodbye $1/;
print "string is now <$string> \n";

        [download]

        /usr/bin/perl -w /tmp/regex.pl 
string is now <Goodbye sailor>

        [download]

        Please note that \1 is somehow deprecated in the substitution part and will cause warnings.

        Its only legitimate use is in the match part for backreference.

        Cheers Rolf

        PS: Je suis Charlie!

[reply]
[d/l]
[select]
        That's how you go if you want to hold the substitution part in a variable.

        my $string = 'Hello sailor';
my $regex = 'Hello (.*)';
my $substitution = '"Goodbye $1"';

$string =~ s/$regex/$substitution/ee;
print "string is now <$string> \n";

        [download]

        /usr/bin/perl -w /tmp/regex.pl 
string is now <Goodbye sailor>

        [download]

        Cheers Rolf

        PS: Je suis Charlie!

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom