in reply to Re: hashes & threads
in thread hashes & threads

Hi,

I did the check you said, and I see fast runtime when while {}. so threads will not help here,Unless I submit a grid job from each thread.

I wonder if a job can be just a subroutine, or it has to be a program..

Guy

Replies are listed 'Best First'.
Re^3: hashes & threads
by Eily (Monsignor) on Jul 28, 2016 at 15:13 UTC

    If by running the empty while loop (but still reading the files) your script runs significantly faster, threads can make a difference (because it's not just time spent waiting for the file to be read). If it does not run faster, parallelisation of any sort is not going to be any more efficient, because all your parallel threads or processes are going to wait their turn to access the device.

    Although if - unlike what I expected - processing the file takes more time than reading it, you should try to see why, because the regular expression should be quite efficient with the ^ making it fail after checking a few characters on an incorrect line. Maybe returning from the function as soon as you have found the "area=" line can be a significant change...

    As a general rule, you should try to find exactly which part of the program is slowing it down (benchmark it) and why before you try to come up with solutions. Overall optimization often is a bad idea, because most of the program runs so ridiculously fast compared to the slowest part that not keeping the simpler version is a waste of development time.