so that the process is heavily IO-bound, but also in part CPU-bound.

In that case, your process isn't either of those. It's just regular mixed processing; and in reality as its extracting a very large volume of data, it probably qualifies as memory-bound.

we usually set the queue to a maximum of about 10 processes running in parallel for our 4-CPU server

With a mixed mode process; that is the sensible choice as allows for greater utilisation of both resources.

But for pure cpu-bound processing, running more processes that there are cpus has the effect of a net increase in overall elapsed time; because it causes more context switches and more cache misses.

In an ideal world of 1 process per cpu, those processes will tend to always occupy the same cpus; thus the caches, especially the L1 caches closest to the cpus, will retain the same data across preemptions. And when preemptions occur; as there are no other processes to be run, the same processes just get another timeslice and pick up right from where they left off.

It's very rarely an ideal world on a modern OS; there are always lots of other systems processes vying for a cpu; but still it is the case that many of those system processes do very little when they get a timeslot -- often just checking one or two flags or ports before relinquishing the rest of their allotment to the next process.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

In reply to Re^2: Useful number of childs revisited [SOLVED] by BrowserUk
in thread Useful number of childs revisited [SOLVED] by karlgoethebier

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.