Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re: Perl::ForkManager does not speed up ATCG calculation !!

by davido (Cardinal)
on Jan 26, 2017 at 21:34 UTC ( [id://1180399]=note: print w/replies, xml ) Need Help??


in reply to Perl::ForkManager does not speed up ATCG calculation !!

Though these are microoptimizations, consider consolidating your tr/// calls to my $Total = $DNA =~ tr/ACGT//;. Make your s/GC/GC/g and s/AT/AT/g calls look like this instead: my $GC = () = $DNA =~ m/GC/g;, and in the while loop, avoid making throwaway copies of the sequence that are only used once.

The bigger issue really is that you're not as processor bound as you might think you are in the portion of the code you shifted into subprocesses. It's likely that if you were to profile (Devel::NYTProf) the code before you converted it to a forking solution, you would discover most of the time is spent making $seqio->next_seq calls in the while loop, and that's one area where there's not much you can do about it.

That call is reading from a stream, and the stream probably has bandwidth limited by the characteristics of the device you are reading from, and forking would be minimally effective or even negatively impactful at the reading stage.

If you are processing many files, you might be able to spread the gathering of those files across several physical devices and then fork a child for each file you wish to process. No single file would run faster, but the overall effect would probably produce savings.


Dave

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1180399]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (4)
As of 2024-04-18 20:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found