Greetings,

Re: Is it handled by an MCE server process with it's own buffering and file I/O will be done on the server process and the result returned via IPC to the hobo?

Yes. The output-class constructs a shared object. Initially, the file handle ($fh) is not defined. This allows MCE::Shared->share to pass the object to the shared-manager, where it will run. The entry point to the shared-manager is the shared-object itself. Method calling and arguments are sent via IPC. The mechanism is fully wantarray aware. Any complex data during transfers involves serialization which is transparent to the application.

Regarding IO position tracking, there is none for shared file-handles constructed via MCE::Shared. The close method was added to the class, so that any pending writes, held in a buffer by Perl, are flushed to disk before exiting; e.g. $iter_o->close().

package Iter::Output;

sub new {
   my ( $class, $path ) = @_;
   my ( $order_id, $fh, %hold ) = ( 1 );

   # Note: Do not open the file handle here, during construction.
   # The reason is that sharing will fail (cannot serialize $fh).

   MCE::Shared->share(
      bless [ $path, $fh, \$order_id, \%hold ], $class
   );
}

sub send {
   ...
}

sub close {
   ...
}
[download]

Regarding Iter::Input, the construction makes a shared object by sending it to the shared-manager process, where it is managed.

package Iter::Input;

sub new {
   my ( $class, $chunk_size, $iterations ) = @_;
   my ( $chunk_id, $seq_a ) = ( 0, 1 );

   MCE::Shared->share(
      bless [ $iterations, $chunk_size, \$chunk_id, \$seq_a ], $class
   );
}

sub recv {
   ...
}
[download]

Replacing MCE::Hobo with threads is possible. One can run MCE::Shared alongside threads and threads::shared.

use threads;

...

test_threads();

sub test_threads {
   my $start = time;
   my @thrs;

   # must save threads to join later
   push @thrs, threads->create('work') for (1..$num_workers);

   # wait for threads to finish
   $_->join for @thrs;

   # close the shared file handle, flushes buffer
   $iter_o->close();

   printf STDERR "testa done in %0.02f seconds\n", time - $start;
}
[download]

Regards, Mario.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.