Re^6: "advanced" Perl functions and maintainability

Quite inconclusive. Considering the what the optimizer does, it highly depends on your data whether index() or a regex is faster. It also depends whether there is a match, where the match is (if any), and whether the string has been studied. Here's some more data:

#!/usr/bin/perl

use strict;
use warnings;

use Benchmark 'cmpthese';

our $string  = "abcd" x 1000;
    $string .= "e";
    $string .= "abcd" x 1000;
our $study   = $string;
our $pass    = "abcde";
our $fail1   = "foo12";
our $fail2   = "abdce";

study $study;

cmpthese(-1, {
    index_pass => 'index($string, $pass)',
    regex_pass => '$string =~ /$pass/',
    study_pass => '$study  =~ /$pass/',
});
print ("\n\n");

cmpthese(-1, {
    index_fail1 => 'index($string, $fail1)',
    index_fail2 => 'index($string, $fail2)',
    regex_fail1 => '$string =~ /$fail1/',
    regex_fail2 => '$string =~ /$fail2/',
    study_fail1 => '$study  =~ /$fail1/',
    study_fail2 => '$study  =~ /$fail2/',
});

__END__
               Rate index_pass study_pass regex_pass
index_pass  38331/s         --        -6%       -69%
study_pass  40960/s         7%         --       -67%
regex_pass 125463/s       227%       206%         --


                 Rate index_fail2 study_fail2 index_fail1 regex_fail2 
+regex_fail1 study_fail1
index_fail2   27306/s          --         -0%        -48%        -56% 
+       -64%        -99%
study_fail2   27307/s          0%          --        -48%        -56% 
+       -64%        -99%
index_fail1   52608/s         93%         93%          --        -15% 
+       -31%        -98%
regex_fail2   61837/s        126%        126%         18%          -- 
+       -19%        -98%
regex_fail1   75918/s        178%        178%         44%         23% 
+         --        -98%
study_fail1 3412032/s      12396%      12395%       6386%       5418% 
+      4394%          --
[download]

Note that with this data, index is slower than a regex. The fastness of 'study_fail1' is explained by the fact that the string we are looking for, 'foo12', contains letters not present in the string - and since a studied string has a histogram attacked of its letter frequencies, no searching needs to be performed at all.

Comment on Re^6: "advanced" Perl functions and maintainability Download Code

Replies are listed 'Best First'.
Re^7: "advanced" Perl functions and maintainability by William G. Davis (Friar) on Dec 13, 2004 at 15:11 UTC
Right, except no one ever uses study, and most Perl hackers still pull out m// even if they only need to search for or one or two constant substrings. Also, you didn't quote meta characters in the pattern strings.	[reply]
Re^8: "advanced" Perl functions and maintainability by itub (Priest) on Dec 13, 2004 at 16:37 UTC
Even without using `study`, the speed advantage of `index` is almost lost when the match is at the end of a long string. `my $string = "this is a string" x 300 . "xyz"; cmpthese(500000, { 'index' => sub { my $res; $res = index($string, "xyz"); }, regex => sub { my $res; $res = $string =~ /xyz/; }, });` [download] Rate regex index regex 97087/s -- -2% index 98619/s 2% -- most Perl hackers still pull out m//... Ok, I agree that `index` is faster in some cases. However, I think there are good reasons for the behavior of most Perl hackers: m// "scales" better in terms of uses. You can use it for the simplest things as well as for very complex things. It is practical and idiomatic. To me, that sounds like a description of Perl itself. If you use it for your constant string to begin with and then you decide you need metacharacters, the change is smaller. Ok, this is a minor advantage, as the change wouldn't be that big anyway. If you are using regular expressions elsewhere in the code, the code looks more consistent, and that makes it more readable. Worring about the speed of `index` vs `m//` may be premature optimization. If you wanted the fastest possible solution you might not want to use Perl in the first place. Even if you want the fastest possible Perl implementation it is always better to make the code correct and readable first, and optimize the hot spots later. I realize that some of this reasons (particularly the last one) agree with your argument for using `for` and `push` instead of `map`. I just wasn't sure that your assertion regarding `index` was correct, as my benchmarks had shown the opposite in the past. Now I see that it depends on the situation. Regarding the readability of `map` vs `for`, I would say that a distinct advantage of `map` is that it documents the purpose of the loop right at the top (when used properly). As soon as you see the `map` keyword you'll know that you are building a list and you'll know where it is being stored; with `for`, you have to wait until you see the `push` to see the true purpose of the loop. Which approach is better depends on the intention of the coder, and I agree that the size of the block may be a factor to consider. The problem is that the exact line between `map` and `for` is blurry, partly a matter of style and personal preference.	[reply] [d/l] [select]
Re^8: "advanced" Perl functions and maintainability by Anonymous Monk on Dec 13, 2004 at 15:43 UTC
Well, you were the one claiming they shouldn't use m// because that's more work for perl than using index, posting a benchmark to back up your claim. I posted a benchmark using different data which shows index losing. index isn't always faster, so Perl hackers aren't "wrong" or even inefficient for using m// over index. Also, you didn't quote meta characters in the pattern strings. Yes I did. If you think I'm wrong, please point out an unquated meta character in one of the pattern strings. If there were unquoted meta characters in the pattern strings, the benchmark wouldn't be fair, would it? index doesn't know metacharacters.	[reply]
Re^9: "advanced" Perl functions and maintainability by William G. Davis (Friar) on Dec 13, 2004 at 16:19 UTC
Well, you were the one claiming they shouldn't use m// because that's more work for perl than using index, posting a benchmark to back up your claim. No, I didn't. You're putting words in my mouth. I claimed that many Perl hackers use m// exclusively, regardless of efficiency, then I posted bechmarks showing that, for constant substrings, index/rindex are usually faster. I posted a benchmark using different data which shows index losing. index isn't* always faster, so Perl hackers aren't "wrong" or even inefficient for using m// over index.* I never claimed it was "always" faster or that people were wrong to use m//. diotalevi tried to use efficiency as a reason to choose map instead of foreach, and I pointed out that most Perl hackers use m// regardless of its efficiency. Some never touch index/rindex. The fact is that efficiency is the last thing on their mind when they make their choice. map and m// generally take less typing, and, I guess, are much cooler, and that's the reason why so many Perl hackers live and die by them. Yes I did. If you think I'm wrong, please point out an unquated meta character in one of the pattern strings. If there were unquoted meta characters in the pattern strings, the benchmark wouldn't be fair, would it? index doesn't know metacharacters. If the scalar containing the substring to search for comes from some extenral source (e.g., supplied to your function as an argument), then you'd have to quote metacharacters in it anyway, which is usaully the only reason why I'd put a scalar inside of a pattern to begin with. Although I guess for this example you could just do away with the scalar altogether and put its contents directly in the pattern.	[reply]
Re^10: "advanced" Perl functions and maintainability by diotalevi (Canon) on Dec 13, 2004 at 21:27 UTC