Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Re^11: Forks, Pipes and Exec (file descriptors)

by diabelek (Beadle)
on Nov 12, 2008 at 04:05 UTC ( #723053=note: print w/replies, xml ) Need Help??

in reply to Re^10: Forks, Pipes and Exec (file descriptors)
in thread Forks, Pipes and Exec

I've switched to using sockets and threads but I've run into an interesting problem. On Windows only (shocker), I am unable to start a thread inside of another thread. I'm unclear on why it stops since depending on where I place the thread->new() it works or doesn't work. I've posted the module code here.

You'll notice in ToolHandler the while loops grabs a command and executes it and then tries to read the queue for the other processes that are running the IO tool. If I place the thread->new() before the _CommandHandler call, it fails to start the new thread and just hangs indefintely.

I'm working on putting together a simple version of the module as a script to see if simplifying it changes anything but if there is any advice on the module it would be appreciated.

  • Comment on Re^11: Forks, Pipes and Exec (file descriptors)

Replies are listed 'Best First'.
Re^12: Forks, Pipes and Exec (file descriptors)
by BrowserUk (Patriarch) on Nov 12, 2008 at 05:45 UTC

      It's a command line IO tool that we use that isn't ours so I changed the name. I just place it in c:\windows since its in the %PATH% and easier to access

      Basically you have the standard output of any IO tool such as the options set, IO/s, errors, warnings, status, etc. If you have ever used iozone or dtstart in Linux, the output would be similar.

      The way the script is right now, the tool doesn't matter since I'm still trying to get the worker threads going that will handle gathering the output from each instance of wintool.exe. Unless there's a better way of implementing it in perl, I'm stuck on why the thread won't start.


      I slowed the script down so I could see whats going on and where its hanging and apparently it hangs completely in the StartIO function when I try to read back the results from the socket. If I change the <$rfh> to this:

      $rfh_select = new IO::Select( $rfh ) or die "Error setting up IO::S +elect: $!\n"; while( !(@ready = $rfh_select->can_read( .1 )) ) { #can_read hangs for .1 seconds so wait for it # need to add some code to stop the loop if we wait to long } sysread( $rfh, $results, 9999999 );
      It will run more or less like it should. Still a few bugs to work out but at least the thread isn't hanging when I ask it to start.

      So the million dollar question that I'm asking is what would a <$handle> in thread A stop a threads->new() from starting thread C & D from thread B?

        so I changed the name.

        Hm. An unfortunate but coincidental 'random' choice of name? (Google to understand what I mean.)

        So the million dollar question that I'm asking is what would a <$handle> in thread A stop a threads->new() from starting thread C & D from thread B?

        To be able to answer that question with any certainty, I'd have to be able to run code that demonstrates the problem with my debugger and other tools. And a part of that would be using the actual executable to see (for example) whether it is buffering its output.

        I'd also have to see what is going on inside the thread that is 'failing to start'. And clearer definition of 'failing to start' wouldn't go amiss.

        However, let me speculate. Does the thread that fails to start attempt to either read or write to the same socket ($handle)?

        Rational: using readline (including via the diamond (<>) operator), will block, even on a socket set non-blocking, until a newline is read. That's the basic definition of readline().

        And, you cannot (under win32 at least), both read from and write to the same socket concurrently on different threads. The OS will serialise concurrent access attempts.

        And if the external process is buffering it's output, then even if every line it writes is always terminated with a newline, the effect of buffering means that when the buffer size, (probably 4096 bytes), is eventually reached, several lines--usually including an incomplete, unterminated last line--will be flushed as a single output.

        If the receiving end of the pipe is doing readlines, then it will block waiting for that incomplete line to be completed. But, the external process may take a while to (or indeed never), fill another buffer load, and so the receiving end just sits and waits. If the other threads you are starting then attempt to write to the socket, they will block until the reading thread completes it's readline. Which may never happen.

        When you set sockets non-blocking, you should avoid buffered IO. Instead use either sysread or recv and syswrite or send.

        That's all speculation (given the absence of good information or code to test), and may only be applicable to win32

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://723053]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (7)
As of 2023-12-01 10:24 GMT
Find Nodes?
    Voting Booth?

    No recent polls found