If you add capturing you can see what it's doing ..

    my $didit=/^
        a         # literal a
        \s*       # optional whitespace
        ((?:
            $re
            \s*
        )+)
        (.*)        # something
        42        # literal 42
    $/x;
    print Dumper [$_, $didit?($1,$2):(undef,undef),  $didit];
[download]

For exampl, with #2 & #6 that gives:

# 2
$VAR1 = [
          'a [foo]  [bar]bla[quux] grmbl42',
          '[foo]  [bar]',
          'bla[quux] grmbl',
          '1'
        ];
# 6
$VAR1 = [
          'a [fo]o] grmbl42',
          '[fo]',
          'o] grmbl',
          '1'
        ];
[download]

The issue is with the "something" .* -- it just sucks up anything bad.