Hi karlgoethebier,

Let's imagine for a minute, the following statement.

$mutex->enter( $shared->set( $_ => $fetch->($_) ) );

1. the worker enters a mutex
   meaning one worker runs solo while inside the mutex
2. then does a fetch on given URL
3. then stores the result into a shared hash
4. finally, leaves the mutex

[download]

The statement above is causing MCE workers to run serially, not parallel. I've gone back to your earlier example here and that looks fine. However for this thread, maybe running solo is what karlgoethebier intended and respecting his decision to do so. Surely, he wanted the code to run parallel ;-).

    mce_loop {
        MCE->yield;

        # run parallel
        my $url = $_; my $result = $fetch->($url);

        # run solo to store the result
        $mutex->enter( $shared->set( $url => $result ) );

        # am back to running parallel
        # ...
    }

[download]

A mutex isn't needed when IPC involves a single trip, typical for the OO interface.

    mce_loop {
        MCE->yield;

        # run parallel, without a mutex
        $shared->set( $_ => $fetch->($_) );

    }

[download]

A mutex is often necessary for a shared hash when constructed via the TIE interface.

    tie my %hash, 'MCE::Shared';
    my $shared = MCE::Shared->hash();
    my $mutex = MCE::Mutex->new();

    $hash{number} = 0;   # 1 trip, store
    $shared->set( number => 0 );   # 1 trip

    # 2 trips fetch and store, needs a mutex
    $mutex->enter( $hash{number} += 2 );

    # 1 trip via the OO interface
    $shared->incrby( number => 2 );

[download]

Regards, Mario

In reply to Re: Yet another example to get URLs in parallel by marioroy
in thread Yet another example to get URLs in parallel by karlgoethebier

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.