However take the string ("foo " x 100)."fo" and try to match it to the regular expression /^(\s*foo\s*)*$/ and Perl 5.6 and up immediately figures out that it doesn't match, while the others won't make significant progress in the projected lifetime of the Sun.

The two issues are connected: to keep the disaster expression from being a disaster, Perl keeps track of extra information during a match to avoid redoing the same work again and again. On most regular expressions this tracking takes a lot of work and buys you little to nothing. However every so often you aren't left wondering why your program unexpectedly froze.

(<[^/]([^>]*[^/>])?>) # xml start tag, e.g. <tag>
|(</[^>]*>) # xml end tag, e.g. </tag>
|(<[^>]*/>) # xml empty tag, e.g. <tag />
|
# Get text at least two characters long that begins with a letter, 
# number, or CJK ideograph, that may also contain some 
# punctuation, and that ends with a letter, number, or CJK 
# ideograph.
  (
    (
      \p{Lu}                # Uppercase Letter
      |\p{Ll}               # Lowercase Letter
      |\p{Lt}               # Titlecase Letter
      |\p{Nd}               # Decimal Digit Number, i.e. 0-9
      |\p{Nl}               # Letter Number, e.g. Roman numerals
      |\p{No}               # Other Number
      |[\x{4e00}-\x{9fa5}]  # CJK Unified Ideographs
      |\x{3007}             # Ideographic number zero
      |[\x{3021}-\x{3029}]  # Hangzhou numerals 1-9
    )
    (
      (
        \p{Lu}
        |\p{Ll}
        |\p{Lt}
        |\p{Nd}
        |\p{Nl}
        |\p{No}
        |[-._:']            # some punctuation
        |[\x{4e00}-\x{9fa5}]
        |\x{3007}
        |[\x{3021}-\x{3029}]
      )*
      (
        \p{Lu}
        |\p{Ll}
        |\p{Lt}
        |\p{Nd}
        |\p{Nl}
        |\p{No}
        |[\x{4e00}-\x{9fa5}]
        |\x{3007}
        |[\x{3021}-\x{3029}]
      )
    )?
  )
[download]

Thanks for taking the time to reformat this. It highlights what I already thought when I glanced at the regexp in the article. I think the writer is confusing capturing and grouping.

I suspect that all the sub-captures for picking up the tricky Unicode stuff are not necessary. In fact, I don't think the alternations for picking up element begin and end tags need to captured either.

They should all be grouped (read: (?:...)) and a single capturing paren around the whole mess. You could then walk down the stream with a while:

  my $token;
  while( $token = $stream =~ /(
       <[^/]([^>]*[^/>])?>
     | </[^>]*>
     | <[^>]*/>
     | (?: ... ) # unicode goop
  )/gx ) {
     print $token;
  }
[download]

Making all those subcaptures available in $1, $2, $3... takes a significant amount of time which could account for Perl's poor showing. (Disclaimer: I have no idea whether Java makes the same distinction between capturing and grouping. If it does so, it's expending as much effort as Perl and my reasoning would be incorrect).

- another intruder with the mooring of the heat of the Perl

Unfortunately there's not much one can say about it without having the file he was parsing on hand.

I can't really say that such a "test" impresses me.