1. Whether the processing will be CPU- or IO- bound depends a lot on the processing. If the processing is complex enough, the IO will be negligible.
  2. If the processes "spend most of their time waiting for disk-IO" and all those images are on the same disk, then starting a lot of processes, all competing for the same disk, is not the best thing to do. Disks nowadays have caches and clever firmware doing read-aheads and other tricks to minimize the need to move the reading heads too much, but with enough processes reading big enough images you can easily render all the caching ineffective and spend time waiting for the heads to move to read the next bit of one of the files. The fact that the tasks are IO-bound doesn't necessarily mean you should start many.
  3. If the processing takes long enough, then starting and destroying a new process for each and every image may not matter much, but it might still help to start eight processes and keep them instead. The easiest solution would be to split the list into eight parts at the start and start a script to process each batch. With thousands of images of a fairly random size, they should all end their work at around the same time, give or take a few images.

Jenda
Enoch was right!
Enjoy the last years of Rome.


In reply to Re^2: Parallel::ForkManager and CPU usage? by Jenda
in thread Parallel::ForkManager and CPU usage? by janmartin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.