Re: Regex combining /(foo|bar)/ slower than using foreach (/foo/,/bar/) ???

Ok /(foo|bar)/ disables a bunch of optimizations that /foo/ || /bar/ can take advantage of. In fact the later can benefit from boyer-moore matching which means it may not have to read the full input stream to tell that the words arent present. However, this wont really scale up, slight modifications of the latter make them also disable the optimisations, and once you start looking at lots of patterns I think you will find that alternate techniques such as Regex::PreSuf, Regexp::Optimizer, Regexp::Assemble or Regexp::List will probably start becomming competitive.

However its worth noting that its very possible that future versions of perl will have optimizations that will make /(foo|bar)/ faster than /foo/ || /bar/. Whatever performance tuning you do will be just that, tuning for your particular enviornment.

---
demerphq

Comment on Re: Regex combining /(foo\|bar)/ slower than using foreach (/foo/,/bar/) ???

Replies are listed 'Best First'.
Re^2: Regex combining /(foo\|bar)/ slower than using foreach (/foo/,/bar/) ??? by JollyJinx (Acolyte) on Feb 18, 2005 at 14:42 UTC
Thanx for enlightment. Using index isn't an option here the program I delivered is an tailored down to verify the original problem. The original problem is that I have THOUSANDS of Regexes to match and replace (not just fixed strings like /foo/ ) in a string that is a few hundred k'Bytes long. Any further suggestions - Jolly	[reply]
Re^3: Regex combining /(foo\|bar)/ slower than using foreach (/foo/,/bar/) ??? by kvale (Monsignor) on Feb 18, 2005 at 16:44 UTC
If your regexes are more complex than simple fixed strings, then it would seem that optimization-destroying alternation would have less of an effect than on speed than the examples above. The regex engine optimises the search for fixed strings down to a fast Boyer Moore algorithm (really fast; I once spent a week trying to tune it with different versions of BM I know - it beat all comers!) but more general patterns need to use the full engine, But if you are so heavily rewriting a string, perhaps it is time to rethink your algorithm. One problem might be that the engine needs to search through the whole string for every search and replace pattern and that search tests the pattern at every position in the string. If you know that the string has a lot of structure and that a given pattern could only match at certain places, attempting a match at every position is a lot of wasted effort. You mentioned XML; if you are replacing particular attributes or elements, it may be a whole lot faster to parse the file using XML::Parser with the lightwieght SAX interface, replacing things as you go. That way the string is only traversed once and replacement becomes, e.g., a simple hash lookup. Update: added a phrase. -Mark	[reply]