Re: is index faster than regexp for fixed text token?

Replies are listed 'Best First'.
Re^2: is index faster than regexp for fixed text token? by sflitman (Hermit) on Jul 06, 2009 at 05:06 UTC
I don't know, I definitely get a big difference. This is with Perl 5.10.0 `Without study: Rate index regexp index 466/s -- -27% regexp 639/s 37% -- With study: Rate index regexp index 467/s -- -54% regexp 1019/s 118% --` [download] Doesn't it make sense that regexp has an advantage in 5.10 since it is building a trie? SSF	[reply] [d/l]
Re^3: is index faster than regexp for fixed text token? by Marshall (Canon) on Jul 06, 2009 at 07:24 UTC
I got similar results...index faster on my machine! `Rate regexp index regexp 848/s -- -74% index 3269/s 286% --` [download] I am unsure why the Ikegami machine does so well on regex. He does have a 64 bit vs 32 bit machine and I don't see why it should make so much difference, but there could be something about that 64 vs 32 bit that speeds things up a lot. I am running Perl 5.10 which is significantly better on regex than Perl 5.8. I suppose there could be other things related to the power of the Ikegami machine like more L2 cache. UPDATE I ran with longer strings to be searched and result does appear to approach the same execution time for both cases: `Rate index regexp index 470/s -- -5% regexp 494/s 5% --` [download] This is an interesting finding. I am not sure why this happens. What I'm guessing is that when the string to be searched is relatively "small", (some ~10+ thousands of chars), index() works better (caveat: in this "search for string X" situation!) because although "dumb" it is fast. But at some point some more "computationally expensive" regex algorithm "gains ground". This is an interesting question for which I have no general heuristic. I think it depends upon length of string to be searched, length of string that we are looking for, the data in each string and perhaps a lot more!	[reply] [d/l] [select]
Re^4: is index faster than regexp for fixed text token? by ikegami (Patriarch) on Jul 06, 2009 at 16:43 UTC
Actually, I seem to remember hearing 5.10's regex engine has a higher initialisation cost (due to the changes to move away from recursion), but maybe I'm thinking of tries (which does have a higher compilation time). I was using 5.8.8, 32-bit threaded. My machine is somewhat old, a P4 without even hyper-threading (which had just come out). I think 5.10.0 gave me similar results, but I'm not sure.	[reply]
Re^5: is index faster than regexp for fixed text token? by Marshall (Canon) on Jul 06, 2009 at 17:12 UTC
Re^2: is index faster than regexp for fixed text token? by sflitman (Hermit) on Jul 06, 2009 at 05:07 UTC
I don't know, I definitely get a big difference. This is with Perl 5.10.0 `Without study: Rate index regexp index 466/s -- -27% regexp 639/s 37% -- With study: Rate index regexp index 467/s -- -54% regexp 1019/s 118% --` [download] Doesn't it make sense that regexp has an advantage in 5.10 since it is building a trie like Regexp::Assemble? SSF	[reply] [d/l]
Re^3: is index faster than regexp for fixed text token? by ikegami (Patriarch) on Jul 06, 2009 at 16:36 UTC
There's no trie involved since there's no alternation.	[reply]