Data passing involves IPC over sockets in MCE including MCE::Shared. It depends on the application whether CPU or network bound. An application polling metrics via SNMP may run with 100 workers and chunk at 300 on a box with 24 logical cores. Each application is unique and not easy to set max workers to some value.

MCE::Map and MCE::Grep closely resembles the native map and grep functions respectively. This means that output order matches input order. If I had to do this again, I would likely merge MCE::Loop and MCE::Flow into one module.

I am unable to provide tips to minimize IPC overhead without knowing what the application is doing. For example, batching updates to a local array or hash. Then populating the shared array or hash using 1 IPC call. This combined with chunking is one way to decrease IPC overhead.

It is quite possible for other AWS instances to be running on the same physical box. Try setting max_workers to 3 or 4 even though 2 CPU cores are allocated for your Ubuntu instance.

Regards, Mario.

In reply to Re: MCE -- how to know which function to use by marioroy
in thread MCE -- how to know which function to use by 1nickt

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.