in reply to Running parallel processes without communication
GNU Parallel http://www.gnu.org/software/parallel/ is made for exactly that purpose. If the output_file is derived from input file (e.g. foo.jpg -> foo.png) you can do this:
cat list | parallel perl perl_code {} {.}.png
If you have a tab separated table of input/output files:
cat list.tsv | parallel --colsep '\t' perl perl_code {1} {2}
If perl_code is a oneliner on the command line you may want to read the section in the man page on QUOTING.
Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
|
|---|