I don't see how that would work. You're proposing fixing a performance issue caused by fork/exec/re-reading a large file from the beginning on each iteration, by doing exactly the same thing but with a virtual machine added in the middle instead of optimised C code? This isn't a perl-specific question you're asking, it's fairly generic. The proposed solutions (e.g., Tie::File) include some perl-specific suggestions, and some that aren't (read the file into memory as a list/array - you can do that in C++ with the STL fairly easily, and Java should make it pretty simple, too), but the general issue is language-agnostic.
Instead, if you read it all into memory, you can use likely just a line to duplicate. Without testing or even compiling:
# do this once. OUTSIDE OF YOUR LOOP.
my @read_source_lines = do {
open my $fh, '<', $read_source or die "Can't read from $read_source:
+ $!";
<$fh>
};
# you may also need:
chomp @read_source_lines; # gets rid of \n's.
# inside the loop, instead of $strx/$str5:
my $str5 = $read_source_lines[$recnum];
print REPORT "$_|$recnum$str5\n";
Assuming you don't start swapping, this should eliminate most of your time. Note that there are better/faster ways to do this, but this will get you most of the benefit for the least amount of effort. Many of those better ways are actually embedded in Tie::File, IIRC (reading only as many lines as is currently needed, continuing from where you left off, maybe you don't need to read the entire file, this may also allow the OS to continue reading the file in the background to fill up your input buffers while you go do other work, that type of thing). | [reply] [d/l] |
Thank you so much for your genuine help and time! Appreciate much!
This piece of code you have suggested( without using Tie::File) , works perfectly fine. now the whole processing takes only a second.
I have heard always perl is very fast, now saw its performance. thanks again for making me a PERL fan too :-)
| [reply] |
please help to amend the script with what you suggest so that i can test (as im quite new to perl scripting). thanks for your understanding and help. | [reply] |