in reply to Using Select and pipes to communicate with threads

Firstly, there is a lot of weirdness in this code.

I don't have a true multi-processor system, so I can't say whether this is faster than just running one big sequential file,

Although you start 5 threads, it never uses more than 25% of my 4-cpu system, and most of the time much less. Indeed, from what I can tell only one thread is ever actually doing anything.

And for the life of me I do not understand what you mean by "running one big sequential file"? How do you "run a file", big, sequential or otherwise.

I just compute primes in ranges in the threads, but the results are all fed directly back to the main thread, without any shared variables involved.

What makes you think that avoiding shared variables is a good thing? What advantages do you think this has over using a simple Thread::Queue?

Apart from that pipes work between threads -- which is a) obvious; b) I've demonstrated here years ago; -- this code seems to take the hard way to achieve something very simple.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^2: Using Select and pipes to communicate with threads
by zentara (Cardinal) on Sep 02, 2011 at 09:25 UTC
    Apart from that pipes work between threads -- which is a) obvious; b) I've demonstrated here years ago; -- this code seems to take the hard way to achieve something very simple.

    All I was trying to demonstrate was the use of IO::Select to collect data from pipes that work between threads and main. I find that a very useful tool to have.

    The primes example was just something I threw together to give the threads something to do, other than sleep

    Can you point out the node where you did this in the past? I would like to see your technique, as I have yet to see it used in any recent nodes about threads and main-child communication.

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
[reply]

      My earliest (that I can find), use of pipe with threads (fork is a thread under win32) is 8 years ago.

      The most recent I found is February this year.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        Your first link is a piped-open command, not a pure pipe, and it dosn't use threads at all. A piped open locks the filehandle to a command, whearas a pure pipe is more flexible.

        Your second link has nothing to do with conventional pipes at all, it states

        Instead of passing a handle to the object back to the parent so that it can control it, try giving the parent a shared variable that indicates what it wants the child to do to the object.

        Neither of these show how to setup conventional pipes to threads, let alone show how to read output from many threads/pipes in a simple select loop.

        Sure, there are many ways to do communication between threads and parent, but neither of your examples show how to use IO::Select to monitor multiple threads whose output is directly fed to pipes, an even more precise, an IO::Pipe object.

        Furthermore, I find your code overly complicated, for example you do 

        # save STDOUT
    open(my $STDOUT_ORIG, ">&", \*STDOUT) or die;

    # connect TO_CHILD_PROC with CHILD_PROC
    pipe(CHILD_PROC, TO_CHILD_PROC);

    # change STDOUT to TO_CHILD_PROC
    # --> STDOUT of child process is set to TO_CHILD_PROC and
    #     this is connected via pipe with CHILD_PROC
    open(STDOUT, ">&", \*TO_CHILD_PROC);

        [download]
        Ugh!!!

        When all that is needed is 

        # in thread
my $wh = $pipe->writer;
$wh->autoflush(1);
print $wh  "whatever \n"  ;

        [download]

        Sorry, but you have not shown any link of yours that uses IO::Select and IO::Pipe in such a clearcut and simple manner, for monitoring multiple threads with 1 loop, as my script does. So I feel it is my right to defend it's virtue and put in Cool Uses for Perl, so others can use the simple technique. Even googling for "perl threads pipe mkfifo" yields few real scripts, and NONE of them combine IO::Select with the simplicity of IO::Pipe.

        So sorry BrowserUk, this time your blusterous hot air dosn't fill the balloon.

        I'm not really a human, but I play one on earth.
        Old Perl Programmer Haiku ................... flash japh
[reply]
[d/l]
[select]

In Section Cool Uses for Perl