Interestiing. Though I think your conclusion regarding "if the speed difference isn't relevant" is wrong. For instance, your test data had 2 "'s in a short string. By the time this gets to 10 "s the speed advantage swings the other way. Even more interesting is that if the string has 0 "s, using substr to avoid the zwa's is twice as fast.

Why would you run it on a string that had no "s. Well, if you're processing large volumes of data into a database from a flat file then you need to check each line as you go. Some will have embedded "s but many won't. You could make the s/// conditional by checking first with m// or index, but just checking causes another pass to be made and costs more than using the simpler regex on the substr.

As Abigail said (again) recently, and has rightly pulled me up for before, you have to check a range inputs before you can make a judgement.

I love Perl. There's always another nuance to explore.

..and remember there are a lot of things monks are supposed to be but lazy is not one of them

OK, I kinda assumed that he gave a "typical" example of an input string, but you're right ofcourse - I should have tested across a wide range of values.

I should actually never have involved benchmarks, since the execution time of the subtitution is likely to be negligable compared to other parts of his application (whatever it may be), and is furthermore dependent on the input strings, on the platform, perl version, and possibly the phase of the moon.

But well, too late for that, the thread is now filled with benchmarks.. my fault :-)

It carries no meaning.

Abigail

Ehm, I benchmarked the code I gave on the example data he gave ofcourse..

my $t = q["string (12" or 1 foot or 1') into 6" MySQL varchar"];
$t =~ s/(?<=.)"(?=.)/""/g;
[download]

versus

my $t = q["string (12" or 1 foot or 1') into 6" MySQL varchar"];
substr($t, 1, -1) =~ s/"/""/g;
[download]

And the number of iterations is shown in the benchmark output (2**20 = 1048576)

Well, all of the discussion about benchmarks set me to wondering which was the most efficient. So here are the actual results. Bear in mind that the benchmarks were only repeated twice and it is a working server. The times where obtained by...

$ticks = time();
....
$ticks = time() - $ticks;
print "$count Records updated in $ticks seconds\n";
[download]

The program is moving data from one MySQL database to another. The subroutine gets passed a hash_ref to a map and a hash_ref from the source database as returned by $query->fetchrow_hashref. The map looks like...

my(%map) = qq(  'AMNT_PAID',            '0.0',
                'AMOUNT',               '{PurchAmt}',
                'DELIVERY_NOTE',        '"{Delivery}"',
                'EXPORTED',             'IF({Exported}, "Y", "N")',
                'FUNDS',                'IF("{CompanyID}"="BCU", "USD"
+, "CAD")');
[download]

The makemap sub returns a string that can be used in the SET portion of an INSERT sql statement. The code snippet follows.

# Create a set list from the data and the data map

sub makemap {

   my($map, $r) = @_;
   my($key, $result, $t, %h, $ndx);
   %h=%$r;
   $result="";

   foreach $key (sort(keys(%$map))) { 
      #print "$key:\t$map->{$key}\n";
      $t = $map->{$key};
        # substitute the actual value for the placeholder fieldname
      $t =~ s/(\{\S+\})/defined(eval("\$h$1")) ? eval("\$h$1") : "NULL
+"/eg;
        # replace any double quotes with escaped doubles
      if ((substr($t,0,1) eq '"') && (length($t) > 2)) {
# best 196 sec
#          $ndx = 1;
#          while( ($ndx = index($t, '"', $ndx)) != -1 and ++$ndx < len
+gth($t)) {
#             substr($t, $ndx++, 0, '"');
#          }
# original 198 sec
#          for ($ndx=1; $ndx < length($t)-1; $ndx++) {
#              if (substr($t,$ndx,1) eq '"') {
#                 substr($t,$ndx,1) = '""';
#                 $ndx++;
#             }
#          }
# substring 204 sec
#         substr($t, 1, -1) =~ s/"/""/g;
# zero width assertion 201 sec 
          $t =~ s/(?<=.)"(?=.)/""/g;
      }
      #print "$t\n";
      $result .= qq($key=$t ,);
   }
   chop($result);       # remove trailing ,
   return($result);
}
[download]

Averaged over two runs each with 17400 records with 750 "pathological" records (with embedded double quotes)the results seem to indicate the iterative method is faster than the regex way but I like the elegance of the regular expression.

Is that really a qq when seeding the map? That doesn't make much sense.

Anyway, given that there's only 4% difference between the fastest and slowest solution, I'd say that it hardly matters. It's probably not the bottleneck in your program anyway. 4% means that on a different machine, or with a different set of data, the order from fast to slow could very well be different.

Abigail