How does the engine know that it has already succeeded with the fourth regex (when pos is already 3)? There must be some mechanism that records that internally in the regex engine.

It might be informative to look at re debugging (although the particular state you're asking about isn't displayed, unfortunately):

use re 'debug';
my $str = "abc";
$str =~ m/.|$/g for 1..5;

__END__

Compiling REx ".|$"
Final program:
   1: BRANCH (3)
   2:   REG_ANY (5)
   3: BRANCH (FAIL)
   4:   SEOL (5)
   5: END (0)
minlen 0 
Matching REx ".|$" against "abc"
   0 <> <abc>                |   0| 1:BRANCH(3)
   0 <> <abc>                |   1|  2:REG_ANY(5)
   1 <a> <bc>                |   1|  5:END(0)
Match successful!
Matching REx ".|$" against "bc"
   1 <a> <bc>                |   0| 1:BRANCH(3)
   1 <a> <bc>                |   1|  2:REG_ANY(5)
   2 <ab> <c>                |   1|  5:END(0)
Match successful!
Matching REx ".|$" against "c"
   2 <ab> <c>                |   0| 1:BRANCH(3)
   2 <ab> <c>                |   1|  2:REG_ANY(5)
   3 <abc> <>                |   1|  5:END(0)
Match successful!
Matching REx ".|$" against ""
   3 <abc> <>                |   0| 1:BRANCH(3)
   3 <abc> <>                |   1|  2:REG_ANY(5)
                             |   1|  failed...
   3 <abc> <>                |   0| 3:BRANCH(5)
   3 <abc> <>                |   1|  4:SEOL(5)
   3 <abc> <>                |   1|  5:END(0)
Match successful!
Matching REx ".|$" against ""
   3 <abc> <>                |   0| 1:BRANCH(3)
   3 <abc> <>                |   1|  2:REG_ANY(5)
                             |   1|  failed...
   3 <abc> <>                |   0| 3:BRANCH(5)
   3 <abc> <>                |   1|  4:SEOL(5)
   3 <abc> <>                |   1|  5:END(0)
END: Match possible, but length=0 is smaller than requested=1, failing
+!
                             |   0| BRANCH failed...
Match failed
Freeing REx: ".|$"
[download]

my question is whether the state that regex uses to determine whether $ was already matched is available to me? ... I would like to understand if the internal structure that has this information ($ matched) is accessible

I have the same question as the others: Why? The regex engine seems to be DWIMing just fine...

I don't know of a way to get at the internal information you're asking about. You can of course simply ask the engine to tell you what it matched, e.g.:

my $str = "abc";
print $str =~ m/(.)|$/g ? "matched! ".($1//'$') : 'no match', "\n"
    for 1..5;

__END__

matched! a
matched! b
matched! c
matched! $
no match
[download]

BTW: Another way to determine the start/middle/end position of a matched substring in a string is by using the  @- @+ regex special variables; see perlvar.

c:\@Work\Perl\monks>perl -wMstrict -le
"my $s = 'foo--bar--baz';
 ;;
 while ($s =~ m{ (\w+) }xmsg) {
   printf qq{matched '$1' %s of '$s' \n},
     $-[1] == 0         ? 'at start'  :
     $+[1] == length $s ? 'at very end' :
                          'in middle';
   }
"
matched 'foo' at start of 'foo--bar--baz'
matched 'bar' in middle of 'foo--bar--baz'
matched 'baz' at very end of 'foo--bar--baz'
[download]

Good point, although in this case those variables also don't allow one to differentiate between the last two matches:

$ perl -MData::Dump -e '$a="abc";for(1..6){ $a=~m/.|$/g; dd \@-,\@+ }'
([0], [1])
([1], [2])
([2], [3])
([3], [3])
([3], [3])
([0], [1])
[download]

Update: Hmm, well they can if one adds capture groups, which is kind of the same workaround as I showed here.

$ perl -MData::Dump -e '$a="abc";for(1..6){$a=~m/(.|$)/g; dd \@-,\@+}'
([0, 0], [1, 1])
([1, 1], [2, 2])
([2, 2], [3, 3])
([3, 3], [3, 3])
([3], [3, 3])
([0, 0], [1, 1])
[download]