perlretut also has prose text to go with the code. This also motivates why it uses (\w\w\w)*?, namely to progress through the string in triplets instead of trying to match at each character position.

There is no such sentence on the explanation.

I linked to perlretut. Going there, I find:

The naive regexp

    ...
[download]

doesn't work; it may match a TGA , but there is no guarantee that the match is aligned with codon boundaries, e.g., the substring GTT GAA gives a match. A better solution is

    while ($dna =~ /(\w\w\w)*?TGA/g) { # note the minimal *?
    print "Got a TGA stop codon at position ", pos $dna, "\n";
    }
[download]

which prints

    Got a TGA stop codon at position 18
    Got a TGA stop codon at position 23
[download]

Position 18 is good, but position 23 is bogus. What happened?

Maybe it was too obvious for me, but a Codon is a nucleotide triplet.

Well of course I agree with Discipulus, because I do love debuggex :)

Here is something that you can try to understand what happens:

use v5.14;
say "With the ?";
'AATCGTTGAATGCAATGACATGAC' =~ /
                                (\w\w\w)*?
                                (?=(?{say "Checking if <$&> is followe
+d by TGA"})) # Print everything that matched before that point
                                TGA/x;
say "Match: $&";
                                
say "\nWithout the ?";               
'AATCGTTGAATGCAATGACATGAC' =~ /
                                (\w\w\w)*
                                (?=(?{say "Checking if <$&> is followe
+d by TGA"})) # Print everything that matched before that point
                                TGA/x;
say "Match: $&";
[download]

In both case, (\w\w\w)+ is a loop that reads three characters at a time. The difference is that in the first case, each times it reads three characters it lets the last part of the regex test the string (check if it is followed by TGA), if the test failed, three new characters are read and the test is ran again. The (\w\w\w)* loop of the second regex though, keeps reading characters as long are there are three characters to read, and it only lets the last part of the regex be checked after it is done, if the test fails, it goes back (backtracks) one iteration, and tries again.

The /g simply memories the position of the last successful match, and starts reading from there on the next attempt.

I think the documentation is a little misleading here. At least, it gives me the impression that the first match (if any) is somehow guaranteed to be valid (because codon-aligned). But that’s true only if, as in the example given, the $dna string happens to contain a valid match somewhere — in which case, it will be found first. But if it doesn’t, the first match is an invalid one:

#! perl
use strict;
use warnings;

while (my $dna = <DATA>)
{
    chomp $dna;
    print "\n\$dna = '$dna'\n";

    while ($dna =~ /(\w\w\w)*?TGA/g)
    {
        print 'Got a TGA stop codon at position ', pos $dna, 
              ', immediately following [', $1, "]\n";
    }
}

__DATA__
ATCGTTGAA
ATCGTTGAATGCAAATGACATGAC
[download]

Output:

 0:10 >perl 1476_SoPW.pl

$dna = 'ATCGTTGAA'
Got a TGA stop codon at position 8, immediately following [CGT]

$dna = 'ATCGTTGAATGCAAATGACATGAC'
Got a TGA stop codon at position 18, immediately following [AAA]
Use of uninitialized value $1 in print at 1476_SoPW.pl line 43, <DATA>
+ line 2.
Got a TGA stop codon at position 23, immediately following []

 0:10 >
[download]

Adding a \G anchor to the regex:

while ($dna =~ /\G(\w\w\w)*?TGA/g)
[download]

fixes the results for both dna strings, because \G means Match only at pos() (e.g. at the end-of-match position of prior m//g) (see “Assertions” in perlre), and initially pos() is set at zero.

<Begin update> choroba is of course correct, anchoring to the start of the string finds only the first match.

while ($dna =~ /^(\w\w\w)*?TGA/g)
[download]

<End update>

Perhaps not Perl documentation’s finest hour. :-)

Hope that helps,

But that means that the regex could also be fixed without recourse to \G, by simply anchoring it to the start of the string:

while ($dna =~ /^(\w\w\w)*?TGA/g)
[download]

With ^ instead of \G, the regex would match only once, even with the /g modifier: because after a succesful match, the next match starts when the previous one ended, and ^ can't match there. With \G, though, you can get all the matches from the loop.

#! /usr/bin/perl
use warnings;
use strict;
use feature qw{ say };

my $dna = join q(), qw( TGA TGA ATG AGA );
for my $regex (qr/(\w\w\w)*?TGA/,
               qr/^(\w\w\w)*?TGA/,
               qr/\G(\w\w\w)*?TGA/,
              ) {
    while ($dna =~ /$regex/g) {
        say "TGA with $regex: ", pos $dna;
    }
}
[download]

($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
[download]

I thought the documentation said that:

... which prints

    Got a TGA stop codon at position 18
    Got a TGA stop codon at position 23
[download]

Position 18 is good, but position 23 is bogus. What happened?

The answer is that our regexp works well until we get past the last real match. Then the regexp will fail to match a synchronized TGA and start stepping ahead one character position at a time, not what we want. The solution is to use \G to anchor the match to the codon alignment: ...

If there is no match at all, I would assume that we are always past the last real match.

By the way, i still don't understand where the regexp backtracks when it fails and test again. (\w\w\w)*?TGA

ATCGTTGAA

Step 1: match the leftmost part 0 times ok, no TGA after, therefore no match. Step2: We start from?

I see! So i'm not the only one! Cheers Athanasius, and thank you very much.

can anybody explain me step by step the procedure that the regexp engine does with the provided string?

Unfortunately, explaining it step by step just takes too long - note, it's not difficult to explain or to understand - it just takes too much typing, since there are a lot of repetitive steps. Maybe someone'll do it anyway? Or maybe not...