in reply to Multiple asynchronous execution of commands

Starting a process, in order to wait for another process, when the main process will still have to wait for the waiting process, is more than a little convoluted.

It also makes life difficult in that the main program doesn't have access to the pid of the subprocess it wants to run/monitor/control, only the pid for the waiting process that it doesn't really want.

threads certainly simplify the scenario, and despite the "deprecation", they still work fine, provided they are enabled at build time.

But, either way, often the simplest of all is to use the piped open, which is asynchronous and returns the pid of the process you want to control/monitor.

The only things to be careful of is if the process you're running produces a lot of output; in which case it will fill the output buffer and block until you read some data from the other end of the pipe periodically to prevent that.

It gets a little more complex if you need the output from multiple processes, as you need to multiplex that reading somehow. (Eg. IO::Select)

If you don't need the output, then redirecting it to null is a possibility, but that reintroduces the problem of the intermediate process (the shell to do the redirection).


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re: Multiple asynchronous execution of commands

Replies are listed 'Best First'.
Re^2: Multiple asynchronous execution of commands
by ibm1620 (Hermit) on Jan 22, 2016 at 21:03 UTC
    I wonder if I was ambiguous - what I want to do is have the main program start several processes, and wait for them all to finish. Not "have the main program start a process that, in turn, starts several processes and waits for them ...". Unless I'm missing a subtlety, I don't think what I'm proposing involves more than a parent process waiting directly upon several child processes.
      Unless I'm missing a subtlety, I don't think what I'm proposing involves more than a parent process waiting directly upon several child processes.

      The "process to wait for a process" arises when you use modules like Parallel::ForkManager as a substitute for threading, as someone else suggested.

      That is, you cannot ask P::FM to start the processes you want directly. Instead, it forks your current program and you then start the process you want to start from that subprocess, whilst the main process continues. It's a messy solution at best.

      The piped-open is asynchronous, thus you can use it to start multiple concurrent processes directly from your main process without problem.

      However, most times you don't only want to start the subprocess, but also monitor (is it still running?) and control (eg. kill it if it takes to long) and at least, eventually receive information that it has ended successfully or otherwise.

      For all of these things, having the pid of the actual subprocess started is necessary; and the piped-open gives you that control. It also gives you access to that subprocesses output; which is easily and economically discarded if not required.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
        That worked nicely. Code snippet:
        my $mysql = q{/usr/local/mysql/bin/mysql --host= [...]}; my @fh; my $i = 0; for my $sql ( q{SELECT id INTO OUTFILE \'/sort/clh/na\' FROM addr}, q{SELECT id INTO OUTFILE \'/sort/clh/ne\' FROM email}, q{SELECT id INTO OUTFILE \'/sort/clh/np\' FROM phone} ) { my $cmd = "echo $sql | $mysql"; open( $fh[$i++], '-|', $cmd ) or die "Failed opening $sql: $!\n"; } say "All spawned - waiting"; for my $i (0..$#fh) { while (defined (my $line = readline( $fh[ $i ] ))) { print "FH$i: $line"; } }
        And as you pointed out, I could use IO::Select if I cared about knowing when each command finished.