Re: fast way to split and sum an input file

There are a number of ways of measuring speed. At the end of the day "time to correct solution" is much more important than "time to execute the program". In this case the (cleaned up) code that you supplied ran in 10 seconds on my system. However the Perl code is about 5 times shorter than the C code so the time to write the Perl code will have been less and the time to debug it will have been much less!

What do you like better:

coding five minutes + 10 seconds run time
coding fifteen minutes + 1 second run time

# Test data generator
use strict;
use warnings;

open outFile, "> data.txt";
print outFile ("0," x 99) . "$_," . ("0," x 10) . "0\n" for (1..100000
+0);
close outFile;


# Sum test
use strict;
use warnings;
use Time::HiRes;

my $val = 0;
my @arr;
my $start = time ();
open inFile, "< data.txt";
while (<inFile>)
    {
    /(?:\d+,){99}(\d+),/;
    $val += $1;
    }
close inFile;
print "Final: [$val] (elapsed time " . (time () - $start) . " seconds\
+n";
[download]

Perl is Huffman encoded by design.

Comment on Re: fast way to split and sum an input file Download Code

Replies are listed 'Best First'.
Re^2: fast way to split and sum an input file by pg (Canon) on Aug 29, 2005 at 21:47 UTC
If this is a program that will be executed repeatedly. I most likely will go with "15 minutes + 1 second". If the performance is important in this case, everybody should be happy to let the guy go with c, as that's the right thing to do. On the other hand, my personal experience tells me that Perl is in general a language with speed. Don't be afraid to declare that even with this incident.	[reply]
Re^3: fast way to split and sum an input file by QM (Parson) on Aug 29, 2005 at 22:01 UTC
For me, performance has to be really important, such as a program that will be running hundreds of times per day. Or for a server, where a small increase in performance means fewer total systems to buy and maintain. The difference between 1 second and 1 minute in the daily grind isn't much at all. Especially considering how long I spend just waiting for some applications to start up in the first place. If the process "multiplier" is only 1 (me, once a day), 1 minute is just long enough for a trip to the soda machine, so I won't mind. Depending on the C compiler, updates can take 10 minutes (compile, link, fix stupid bugs, repeat) compared to Perl. I once had a class project to write 1/3 of a trading system (along with my 2 teammates, who wrote the other 2/3). They used C, I used Perl. One of them implemented the interface wrong, so our project didn't work. We had 20 minutes to deadline, and he was going to need 2 hours to find the bug and fix it. Instead, I changed my Perl version to handle his mistake, and we turned it in 5 minutes after we discovered the problem (including testing). Speed of execution is rarely a requirement. -QM -- Quantum Mechanics: The dreams stuff is made of	[reply]