Re: An optimization of last resort: eliminate capturing from your regexps
by demerphq (Chancellor) on Jul 11, 2006 at 08:43 UTC
|
I wonder... A switch could be added that would tell perl that you commit to not modifying the string before utilizing $1. Then the memcopy would be unnecessary. That way you could say
$str =~ /Hulk hate (\w+)/k;
And get the same effect. Of course the demons that would fly out of your nose if you said:
if ($str =~ /Hulk hate (\w+)/k) {
$str='demons!';
print $1; # could and probably would throw fatal error.
}
Would be your problem...
BTW, if it isnt clear, this is the reason the memcpy is needed.
Alternatively, perhaps magic could be introduced to $str so that the memcpy would only happen if $str was modified while the match vars pointed at it...
A last possibility would be to not do the memcpy if the string was RO. Then you could by hand readonly the string, do the match, use $1 and then when and if you needed to modify the string undo the RO.
I guess another variant could be that the /k would result in no copy, and an understanding that accessing $1 et all would be a fatal error while that regex was the last used. However the @- @+ arrays would be populated. Its just the user would be expected to do the substring operations by hand.
---
$world=~s/war/peace/g
| [reply] [d/l] [select] |
|
Making that a flag on Perl is a horrible idea. Quick, I want to speed up my program, can I use that flag? I dunno, do you want to go through all of the modules you import and didn't write?
Make that a flag on the regular expression instead. Then people can turn it on when they find a critical part of their code to optimize. And don't have to worry about looking through modules written by other people.
As tye pointed out, I misread. Sorry.
| [reply] |
|
Am I misunderstanding you or did you just not notice the second paragraph:
$str =~ /Hulk hate (\w+)/k;
Note that demerphq has added /k to his regex. I don't see demerphq proposing any per-script or per-invocation flag that you seem to have imagined.
| [reply] [d/l] |
|
If a pattern doesn’t contain (?{}) or (??{}) bits, then $str cannot change during a match. So in that case it would be feasible to postpone the memcpy until right after the match (before the regex engine returns) and memcpy only the matched bits. That way, all regexen which don’t run Perl code would automatically avoid unnecessary copying. I think that would be a worthwhile patch.
Makeshifts last the longest.
| [reply] |
|
| [reply] |
|
|
Re: An optimization of last resort: eliminate capturing from your regexps
by Hue-Bond (Priest) on Jul 11, 2006 at 08:22 UTC
|
my $str = 'Hulk hate classless system with means of manufacturing give
+n to working class! Hulk crush puny Marxists!';
if ($str =~ /Hulk hate \w+/) {
print "\$-[0] <$-[0]> \$+[0] <$+[0]>\n";
my $word = substr $str, $-[0] + 10, $+[0] - $-[0] + 10;
print "word <$word>\n";
}
__END__
$-[0] <0> $+[0] <19>
word <classless system with means o>
That +10 should be -10.
Anyway, the reason I'm replying is not this, but to suggest some typeglob aliasing to make the code a tiny bit easier to read:
*starts = \@-;
*ends = \@+;
our (@starts, @ends);
my $str = 'Hulk hate classless system with means of manufacturing give
+n to working class! Hulk crush puny Marxists!';
if ($str =~ /with means of \w+/) {
print "\$-[0] <$-[0]> \$+[0] <$+[0]>\n";
my $word = substr $str, $starts[0] + 14, $ends[0] - $starts[0] - 1
+4;
print "word <$word>\n";
}
__END__
$-[0] <27> $+[0] <54>
word <manufacturing>
| [reply] [d/l] [select] |
|
| [reply] |
Re: An optimization of last resort: eliminate capturing from your regexps
by graff (Chancellor) on Jul 11, 2006 at 05:26 UTC
|
So, is it really the case that using @- and @+ like you do here does not cause the runtime penalty that is imposed by using the "match" variables ( $`, $&, $') ? If so, that's good to know...
perlvar (in my current 5.8.6) still has the warning about the impact of using any of those three match variables in a script, but that impact is not mentioned at all in the sections about @+ and @- (however the latter does indicate how to use these arrays to get the same values as what the match variables would give you). | [reply] [d/l] |
|
substr($something, $-[$digit], $+[$digit] - $-[$digit]);
Part of the reason for this is that internally perl doesnt populate $1 et all with copies when it runs the regex as when backtracking it could have to recopy the contents of these vars many many times. Wheras updating the appropriate offsets is cheap and constant time.
---
$world=~s/war/peace/g
| [reply] [d/l] |
|
| [reply] |
|
Correct, really, @- and @+ do not impose the performance penalties incurred with $', $&, and $`. Think of the @- and @+ variables as simply keeping track of where something was seen, whereas the 'match' variables keep copies of what was seen. It's quicker to bookmark page nine than it is to memorize it. ;) (ok, that's a bit of a simplistic exaggeration, but it should get the idea across)
| [reply] |
|
Hopefully to be somewhat clearer, $`, $&, and $' are especially bad because once they've been seen by Perl, all regexes will do extra work.
Capturing in a regex imparts a performance hit because it means that a copy will be made of the string that the regex is being applied to (which makes it a worse performance hit when matching against really large strings -- one of the worst cases being running a lot of little regexes with capturing against the same huge string, something a parser is likely to do). The performance hit of capturing only applies to the regex that does capturing and most of the time it isn't enough that you'd notice.
@- and @+ are always set by regexes and whether you use them or not doesn't have any performance impact at all.
I'm curious why the regex engine doesn't just copy out the parts that were captured and only copy them after the regex has finished. That seems like a "best of both worlds" solution. Though, I think /k would be really cool, especially if you could use it with:
@matches= $string =~ /$regex/gk;
to make it so it doesn't matter whether $regex contained capturing parens or not. (:
Finally, this benchmark shows that the per-regex performance hit from $`, $&, and $' is the same hit from capturing parens:
#!/usr/bin/perl -w
use strict;
use Benchmark qw( timethis cmpthese );
my %t;
my $pref= "no&";
my $str= "match" . ( "garbage" x 500 );
for( 0, 1 ) {
$t{$pref."?:"}= timethis( -3, sub {
$str =~ /(?:match)+/;
} );
$t{$pref."()"}= timethis( -3, sub {
$str =~ /(match)+/;
} );
eval '$&';
$pref= "amp";
}
cmpthese( \%t );
__END__
Rate amp() no&() amp?: no&?:
amp() 439813/s -- -0% -1% -81%
no&() 441612/s 0% -- -0% -81%
amp?: 443563/s 1% 0% -- -81%
no&?: 2354611/s 435% 433% 431% --
But, remember: $`, $&, and $' are evil while capturing parens are usually the best solution when you need them and usually don't cause a noticeable performance penalty.
| [reply] [d/l] [select] |
|
True. Using @- and @+ does not invoke the $`, $&, $' penalty. The $@ penalty is just the memcpy penaly I mentioned except it is enabled for all regexps, even the non-capturing ones. Any regexps that already use capturing can't be penalized anymore and aren't any slower because of it.
| [reply] |