Hello all,

I've read through the docs and made some experiments with basic usage of MCE, but I'm not sure if I'm barking up the wrong tree.

I'm unclear on:

I have an arrayref of hashrefs, and am outputting an arrayref of hashrefs. Processing each hashref is quite slow: takes about 0.1s. There are 7,500 hashes in the arrayref; that could grow to some tens of thousands.

The code is running on an Ubuntu AWS instance whose lscpu outputs: 

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                2
On-line CPU(s) list:   0,1
Thread(s) per core:    2
Core(s) per socket:    1
Socket(s):             1

[download]

The MCE manager is splitting the array into 25 chunks using 'auto'.

I am seeing almost no difference in time taken to execute using MCE versus a sequential foreach loop, in fact the sequential loop appears faster, which I would not have expected.

Although the CPU usage looks quite different: 

Benchmark: timing 5 iterations of MCE loop       , MCE map        , Se
+quential loop...
MCE loop       : 83 wallclock secs ( 3.28 usr  0.19 sys + 30.94 cusr 2
+4.89 csys = 59.30 CPU) @  0.08/s (n=5)
MCE map        : 75 wallclock secs ( 4.48 usr  0.28 sys + 41.29 cusr 3
+7.93 csys = 83.98 CPU) @  0.06/s (n=5)
Sequential loop: 76 wallclock secs (37.79 usr + 28.91 sys = 66.70 CPU)
+ @  0.07/s (n=5)

[download]

Am I missing something obvious? Or non-obvious? Doing something wrong? Can anyone shed any light please?


The way forward always starts with a minimal test.

In reply to MCE -- how to know which function to use by 1nickt

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.