Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Please know that the results on this post doesn't count. I've overclocked a system on purpose, mainly for simulating a fast machine in the future. On an 8-core box (total 16 logical cores), I ran the same tests to better understand the impact between the three implementations. The box is overclocked to 4.0 GHz. What is interesting to me is not how fast but rather the impact from running many workers.

Baseline testing involves 4 workers and runs about 2x faster than my laptop. No surprises there due to running at 4 GHz and not from a Linux VM.

Update: Added results for MCE::Inbox which will ship with MCE::Shared 1.827 on release day. It is Inbox2 with optimizations.

# 4 GHz, CentOS 7 ( Perl v5.16.3 ) $ perl foo1.pl | wc -l # Foo::Inbox duration: 0.441 seconds 50000 $ perl inbox.pl | wc -l # MCE::Inbox w/ blocking capability duration: 0.422 seconds 50000 $ perl foo2.pl | wc -l # MCE::Shared->queue duration: 0.645 seconds 50000 $ perl foo4.pl | wc -l # Thread::Queue duration: 1.544 seconds 50000

Next is 128 workers retrieving 1.6 million messages.

# 4 GHz, CentOS 7 ( Perl v5.16.3 ) $ perl foo1.pl | wc -l # Foo::Inbox duration: 12.685 seconds 1600000 $ perl inbox.pl | wc -l # MCE::Inbox w/ blocking capability duration: 15.939 seconds 1600000 $ perl foo2.pl | wc -l # MCE::Shared->queue duration: 21.533 seconds 1600000 $ perl foo4.pl | wc -l # Thread::Queue duration: 90.015 seconds 1600000

Unfortunately, threads may not scale linearly, especially when involving locking. In top, I see the process peak at 306% max while running. It's far from the 1600% mark hardware limit. On the other hand, MCE::Inbox scales nicely considering it does blocking as well.

For the 1.6 million test, I added a loop to generate more names, 128 total.

my @names = shuffle qw/ Barny Betty Fred Wilma /; foreach my $i ( 1 .. 5 ) { push @names, map { $_.$i } @names; print scalar(@names), "\n"; } __END__ 8 16 32 64 128

There was one other change and replaced 4167 with 98 because there are now 128 workers, not 4.

... # send greeting again $inbox->send($name, \@names, 'Hello') if $count < 98; # eventually stop benchmarking last if ++$count == 12500; ...

Regards, Mario


In reply to Re^3: Child process inter communication by marioroy
in thread Child process inter communication by smarthacker67

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2024-04-19 13:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found