in reply to Fast reading and processing from a text file - Perl vs. FORTRAN

Having profiled you code line by line two different ways, it seems that almost all of the time is spent performing IO. Either reading from or writing to files. The only executable line that shows any significant cpu ot elapsed time usage is the the line

92:         my @array = split(" ", $in);

Which I don't see any easy way to optimise.

Of course, the IO charecteristics of my system may be significantly different from the system you are running on, though I notice you are on a Win32 sytem too, so it shouldn't be so different.

As for why the FORTRAN program is so much faster, assuming that it is doing the same processing,IO and running on the same hardware (which is an open question?), then my best guess is that the FORTRAN IO uses larger buffers and asynchronous reads and writes. I know of at least one PC-based FORTRAN compiler that does this.

It is possible to use threads and other techniques to acheive this using perl without radically altering the structure of your existing program, but it isn't trivial to do. Whether you would come close to getting the kind of speed up you would need is very speculative.

This is the output from using Devel::SmallProf on your code. I've massaged the output to just show the significant lines in decreasing order of cpu usage.

cpu/iter iters wall s cpu s line 0.4281 1 0.428139 0.000000 19:close FAIL_FLAG_QUAD4; 0.0034 296 0.999800 0.110000 121: print FAIL_FLAG_QUAD4 + ("$Element_ID 0.0011 888 0.999800 0.191000 112: $Flag_for_elem_id_line + = 0; 0.0003 3257 0.999800 0.623000 62: if (($in =~ /^1/) && ($in + =~ /MSC/) && 0.0003 30192 8.998197 6.425000 92: my @array = split(" + ", $in); 0.0003 13912 3.999199 3.344000 115: print Q4CE_IN ("$Eleme +nt_ID 0.0003 13912 3.999199 2.245000 116: $Flag_for_long_line = +0; 0.0003 13912 3.999199 3.103000 118: print Q4FI_IN ("$Eleme +nt_Id_Line 0.0003 13912 3.999199 2.973000 120: if( ($size == 2) && ( +(defined 0.0002 31672 5.998798 6.290000 88: $in = (<FILE>); 0.0002 31672 4.998999 5.005000 89: chomp($in); 0.0002 30192 5.998798 4.405000 93: my $size = @array; 0.0002 30192 6.998598 5.549000 95: if($size == 5){ 0.0002 13912 2.999399 2.341000 103: $Ply_Id = $array[0]; 0.0001 13912 1.999599 2.293000 104: $Failure_Index_1 = $ar +ray[1]; 0.0001 13912 0.999800 2.935000 106: $Flag_for_long_line = +1; 0.0000 1 0.000000 0.000000 5:my @FileArray = ("junk.txt") +; 0.0000 1 0.000000 0.000000 7:open(Q4FI_IN, ">QUAD4FI.txt" +) or die "Unable 0.0000 1 0.000000 0.010000 9:open(Q4CE_IN, ">QUAD4CE.txt" +) or die "Unable

I noticed a couple of errors in your code, but these have been noted above.

One question? Are you guys paid by the keystoke? :^)

Those have to be the longest variable names I've come across in a while.


Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller

Replies are listed 'Best First'.
Re: Re: Fast reading and processing from a text file - Perl vs. FORTRAN
by ozgurp (Beadle) on May 26, 2003 at 07:11 UTC
    Thanks to everyone who responded.

    Ozgur