in reply to Interesting Perl/Java regexp benchmarking

Here is my attempt at commenting the perl regex Bray uses. Note that Unicode sub-properties are heavily used. 

(<[^/]([^>]*[^/>])?>) # xml start tag, e.g. <tag>
|(</[^>]*>) # xml end tag, e.g. </tag>
|(<[^>]*/>) # xml empty tag, e.g. <tag />
|
# Get text at least two characters long that begins with a letter, 
# number, or CJK ideograph, that may also contain some 
# punctuation, and that ends with a letter, number, or CJK 
# ideograph.
  (
    (
      \p{Lu}                # Uppercase Letter
      |\p{Ll}               # Lowercase Letter
      |\p{Lt}               # Titlecase Letter
      |\p{Nd}               # Decimal Digit Number, i.e. 0-9
      |\p{Nl}               # Letter Number, e.g. Roman numerals
      |\p{No}               # Other Number
      |[\x{4e00}-\x{9fa5}]  # CJK Unified Ideographs
      |\x{3007}             # Ideographic number zero
      |[\x{3021}-\x{3029}]  # Hangzhou numerals 1-9
    )
    (
      (
        \p{Lu}
        |\p{Ll}
        |\p{Lt}
        |\p{Nd}
        |\p{Nl}
        |\p{No}
        |[-._:']            # some punctuation
        |[\x{4e00}-\x{9fa5}]
        |\x{3007}
        |[\x{3021}-\x{3029}]
      )*
      (
        \p{Lu}
        |\p{Ll}
        |\p{Lt}
        |\p{Nd}
        |\p{Nl}
        |\p{No}
        |[\x{4e00}-\x{9fa5}]
        |\x{3007}
        |[\x{3021}-\x{3029}]
      )
    )?
  )

[download]

Replies are listed 'Best First'.
Re^2: Interesting Perl/Java regexp benchmarking (capturing vs. grouping)
by grinder (Bishop) on Aug 24, 2004 at 07:06 UTC

    Thanks for taking the time to reformat this. It highlights what I already thought when I glanced at the regexp in the article. I think the writer is confusing capturing and grouping.

    I suspect that all the sub-captures for picking up the tricky Unicode stuff are not necessary. In fact, I don't think the alternations for picking up element begin and end tags need to captured either.

    They should all be grouped (read: (?:...)) and a single capturing paren around the whole mess. You could then walk down the stream with a while:

      my $token;
  while( $token = $stream =~ /(
       <[^/]([^>]*[^/>])?>
     | </[^>]*>
     | <[^>]*/>
     | (?: ... ) # unicode goop
  )/gx ) {
     print $token;
  }

    [download]

    Making all those subcaptures available in $1, $2, $3... takes a significant amount of time which could account for Perl's poor showing. (Disclaimer: I have no idea whether Java makes the same distinction between capturing and grouping. If it does so, it's expending as much effort as Perl and my reasoning would be incorrect).

    - another intruder with the mooring of the heat of the Perl

[reply]
[d/l]
[select]

In Section Meditations