Benchmark: timing 30 iterations of REGEXP, SPLIT...
    REGEXP: 10 wallclock secs ( 9.73 usr +  0.25 sys =  9.98 CPU)
     SPLIT: 47 wallclock secs (47.20 usr +  0.28 sys = 47.48 CPU)
[download]

The script didn't pass muster with "-w" when I was trying to print matching lines to "NUL" (file handle hadn't been opened). I changed it simply to count matching lines. "-w" is now happy. (Note to self: something else for later study: how to print only to "NUL" w/out complaint from "-w".)

The source (CSV) file is 13,576 lines long (1,703,397 bytes). Each record has 12 fields; the average length per record is 124 characters. The task is to print only lines whose fourth fields contain "MAPI".

use strict;
use Benchmark;

timethese( 30, {
      REGEXP => 'UsingRegExp',
      SPLIT  => 'UsingSplit'
    }
);

sub UsingRegExp {
    my $file = 'r:\csv\test.csv';
    my $field;
    my $count = 0;
    open FH, $file or die "\n $file: $!\n";
    while ( <FH> ) {
        #  WANT 4TH FIELD. (NOTE: SOME FIELDS _MIGHT_ BE EMPTY.)
        ($field) = /^[^,]+,[^,]*,[^,]*,\s*([^,]+)\s*,/;
        $count++ if lc($field) eq "mapi";  #  IGNORE CASE
    }
    close FH or die "\n $file: $!\n";
}

sub UsingSplit {
    my $file = 'r:\csv\test.csv';
    my @record;
    my $count = 0;
    open FH, $file or die "\n $file: $!\n";
    while ( <FH> ) {
        @record = split /\s*,\s*/;
        $count++ if lc($record[3]) eq "mapi";  #  IGNORE CASE
    }
    close FH or die "\n $file: $!\n";
}
[download]

I prefer this to whiles:

<code> map{$r=[split (regex here)];$c+=(lc($r->[9]) eq "mapi")}<FH>; <code>

I haven't tested this works but I hope you get my drift.

Brother Frankus.

It's the crudest possible test, of course. But why bother with further processing if the string doesn't appear anywhere in the current line?

With that statement added, there was a considerable difference in the benchmarked results:

Benchmark: timing 30 iterations of REGEXP, SPLIT...
    REGEXP: 14 wallclock secs (14.35 usr +  0.19 sys = 14.54 CPU)
     SPLIT: 17 wallclock secs (16.01 usr +  0.28 sys = 16.29 CPU)
[download]

The question about caching is a good one. I can't say for certain that caching didn't play a part in the performance boost (or in that case I guess I should say a perceived performance boost).

I later re-created the business end of the routine on another computer and timed both approaches. The results were similar. Haven't yet benchmarked it. I'm still, argh, a bit hazy on exactly how to use Benchmark. But never mind "hazy"--I will get make my way through the haze and try it. (To date the two approaches have been timed using only the 4nt command processor's own timer function--far from exact, to be sure.)

One noticeable difference between the regular expression I used and the one you used in your example here: I had only one set of parens in it. I don't know if this is likely to make a big difference in performance.

Thanks for the feedback, folks.

Only using one capturing paren will improve the performance of your regex as it will not be forced to do as much backreferencing. In playing around with this, I managed to optimize the split by breaking it into a minimal number of segments. In all cases, with my example, split significantly outperformed the regex.

#!/usr/bin/perl -w
use strict;
use Benchmark;
use vars qw($myvar $result $a $b $c $d);
$myvar = "one,two,three,four";

timethese(1000000, { 
    Regex  => '$a=$1, $b=$2, $c=$3, $d=$4 if $myvar   =~ /^[^,]+,([^,]
++),[^,]+,[^,]+$/',
    Split1 => '$result = (split /,/, $myvar)[1]',
    Split2 => '$result = (split /,/, $myvar, 4)[1]',
    Split3 => '$result = (split /,/, $myvar, 3)[1]'
});

Benchmark: timing 1000000 iterations of Regex, Split1, Split2, Split3.
+..
     Regex: 26 wallclock secs (25.75 usr +  0.00 sys = 25.75 CPU)
    Split1: 16 wallclock secs (16.31 usr +  0.00 sys = 16.31 CPU)
    Split2: 16 wallclock secs (16.15 usr +  0.00 sys = 16.15 CPU)
    Split3: 13 wallclock secs (12.74 usr +  0.00 sys = 12.74 CPU)
[download]

Note the whopping improvement in performance of Split3. In my benchmark, it's approximately twice as fast as the regex.

Cheers,
Ovid

But your comparison isn't fair. You let the regex do way much work than needed. There's no need to parse the entire line, and only assign if there are exactly four fields - you aren't doing that for the split cases either. Also, you only have one set of parens, yet you do four assignments. Picking a simpler regex, and doing just one assignment improves the speed with 50%!

$a=$1 if $myvar =~ /^[^,]+,([^,]+)/
[download]

Still not as fast as the split, but it shows that proper Benchmarking is an art.

-- Abigail