We were trying to speed up a process where we had to call a system command about 40 times, and process the output. My first try at a sample program had the close and remove() lines below swapped, and this caused badness in that some of the processes were never read from (it seems to be bad to remove a closed filehandle from an IO::Select object). Also, after carefully reading the IO::Select documentation, I saw that you could pass an arrayref to the add() method (never noticed that before!), thereby saving some extra info with the handle in the select object, which came in handy. The below is my resulting "Hello World" example:
#!/usr/bin/perl

use warnings;
use strict;

use IO::Select;

# Number of simultaneous processes
my $n = 5;

# List of things to process
my @p = (1..20);

my $sel = IO::Select->new();
for my $pr (splice @p, 0, $n) {
  start_ps($sel, $pr);
}

my $cnt = 1;
while ( my @ready = $sel->can_read() ) {
  for my $h (@ready) {
    local $_;
    my ($fh, $i) = @$h;
    while (<$fh>) {
      print "$i: $_";
    }
    # Remove, then close!!!
    $sel->remove($fh);
    close $fh;
    start_ps($sel, shift(@p)) if @p;
    $cnt++;
  }
  print "-------------\n";
}

sub start_ps {
  my ($t, $p) = @_;
  my $pid = open(my $fh, "-|");
  die "Can't fork: $!" unless defined $pid;
  unless ($pid) {
    my @cmd = qw(/bin/echo Hello World);
    exec @cmd;
    die "Could not exec @cmd: $!";
  }
  $t->add([$fh, $p]);
}

[download]

Replies are listed 'Best First'.
Re: Multiprocessing with IO::Select
by targetsmart (Curate) on Jan 31, 2009 at 09:52 UTC
    Thanks very much, I have did a lot of socket programs, but I have never noticed, thanks for pointing out
    From IO::Select man page 
           add ( HANDLES )
           Add the list of handles to the "IO::Select" object. It is t
+hese values that will be returned when an event occurs. "IO::Select" 
+keeps these values in a cache which is indexed by the "fileno" of the
+ handle, so if more than one handle with the same "fileno" is specifi
+ed then only the last one is cached.

           Each handle can be an "IO::Handle" object, an integer or an
+ array reference where the first element is an "IO::Handle" or an int
+eger.

    [download]
    Each handle can be an "IO::Handle" object, an integer or an array reference where the first element is an "IO::Handle" or an integer. is the key point I have noticed.
    So far I have been using some extra hashes(where the keys would be fileno(handle)) to maintain the handle specific information, now it won't be required.
    Vivek
    -- In accordance with the prarabdha of each, the One whose function it is to ordain makes each to act. What will not happen will never happen, whatever effort one may put forth. And what will happen will not fail to happen, however much one may seek to prevent it. This is certain. The part of wisdom therefore is to stay quiet.
[reply]
[d/l]

Back to Meditations