Sync has asked for the wisdom of the Perl Monks concerning the following question:

Am running ActiveState perl v5.8.7 on Windows XP Pro. Tried the following test script. The idea here is to run two parallel threads that process a common queue of todo items. The processing however, requires running some external executables. Had first tried the backtic as I'd like to get the output. Was hanging so I tried some other combinations.

Finally arrived at the following test script. It sorts a small file called testfile with a couple of words to sort in it (anything will do). I can run it on Debian Linux (using the appropriate $XTCMD definition uncommented) and all three settings of $EXEMODE work fine, and pretty much identically.

On Windows XP however, only $EXEMODE = 0 works. The others both hang on the external call.

I'd really like to avoid the temporary file technique that I'm limited to with system(), and use EITHER the backtic or the popen version (or something else that would work if I'm not aware of it)...

Any idea what's happening here? Windows pseudo-fork anomalies perhaps? Any other known alternatives?

-- Sync

#!/usr/bin/perl -w use strict; use threads; use threads::shared; # EXEMODE: 0 - system, 1 - bactic, 2 - popen # Try all three to compare my $EXEMODE = 0; #my $XTCMD = "/usr/bin/sort testfile"; # linux test my $XTCMD = "c:\\windows\\system32\\sort.exe testfile"; # windows tes +t my $queueindex : shared; share $queueindex; my $thearg = 0; my $qsize = 10; $queueindex = 0; sub inc_qindex { # print "Locking index\n"; lock $queueindex; $queueindex++; # print "Returning index $queueindex\n"; return $queueindex; } sub dothread { my ( $v, $m, @m ); my $taskctr = 0; my $tmpfile = "tmp" . $thearg . ".out"; while (1) { $v = &inc_qindex; # get next queue item last if ( $v > $qsize ); # end of queue print "I am thread [" . $thearg . "] index = [" . $v . "]\n"; if ( $EXEMODE eq 0 ) # this works OK { system("$XTCMD >$tmpfile"); open XT, "$tmpfile" or die "$!: Can't open $tmpfile!"; @m = (<XT>); close XT; $m = join( "", @m ); print $m; # will see this } if ( $EXEMODE eq 1 ) # this hangs { $m = `$XTCMD`; print $m; # never see this } if ( $EXEMODE eq 2 ) # this also hangs { open XT, "$XTCMD|" or die "$!: trying to run $XTCMD!"; @m = (<XT>); close XT; $m = join( "", @m ); print $m; # never see this } $taskctr++; } return $taskctr; } sub test_threaded { my ( $ta, $tb, $r ); $thearg = 1; $ta = threads->new("dothread"); $thearg = 2; $tb = threads->new("dothread"); $r = $ta->join; print "Thread 1 returned [$r]\n"; $r = $tb->join; print "Thread 2 returned [$r]\n"; } #dothread; # test it unthreaded to make sure it works test_threaded;
  • Comment on Behavior of threads on XP-- system() works, backtic & popen don't...
  • Download Code

Replies are listed 'Best First'.
Re: Behavior of threads on XP-- system() works, backtic & popen don't...
by BrowserUk (Patriarch) on Jun 13, 2006 at 01:40 UTC

    All three work for me using AS817 under XP using the following test script.

    #! perl -slw use strict; use threads; sub test { system qq[ c:/windows/system32/sort.exe test.dat > sorted.dat +]; open F, '<sorted.dat' or die $!; print "Via system:\n", do{ local $/; <F> }; close F; print "Via backticks\n", `c:/windows/system32/sort.exe test.da +t`; open F, "c:/windows/system32/sort.exe test.dat |" or die $!; print "Via pipe\n", <F>; close F; } async( \&test )->join;

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Interesting. A couple of things:

      First, thanks for the response.

      Second, it took me some effort to find your response as I had posted the original to "seekers of perl wisdom" but never actually saw it show up anywhere except when I login and look at my "Nodes You Wrote." I finally did a "super search" of everything for "threads XP" in the title and finally discovered your reply-- I still don't know where this discussion thread is actually showing up, as the "super search" didn't seem to tell me *where* it found what it found...

      And finally, I'm using AS 813, and your example works for me too. However, what it doesn't do is run multiple threads in parallel-- it just runs a single thread. So, I tweaked it a bit and came up with the attached version of your program. I was running into errors due to sort not being able to access the test.dat file in each thread due to collisions, so I created 4 copies of the test.dat file (test1-4.dat) and also use the scheme for the sorted.dat output file so that the threads won't be using each others files. I then run async 4 times to spawn off 4 threads. I also added a sleep in the thread (probably not necessary) to make sure all 4 threads are running before any of them start doing their thing. After that, the first thread's system() call works, and then the whole thing hangs. If I only run one thread in "spawnem" then it works OK...

      #! perl -slw use strict; use threads; sub test { my $t = shift; my $cmd = "c:/windows/system32/sort.exe test$t.dat"; print "Thread $t\n"; # show things are in motion... sleep 1; # kludge to insure all threads are up... system "$cmd >sorted$t.dat"; open F, "<sorted$t.dat" or die $!; print "Via system:\n", do{ local $/; <F> }; close F; print "Via backticks\n", `$cmd`; open F, "$cmd|" or die $!; print "Via pipe\n", <F>; close F; } sub spawnem { my $t1 = async{ \&test(1); }; my $t2 = async{ \&test(2); }; my $t3 = async{ \&test(3); }; my $t4 = async{ \&test(4); }; $t1->join; $t2->join; $t3->join; $t4->join; } &spawnem;

        Strange. I created the four data files required and ran your code as is and it produced the following output (I've wrapped the numbers onto single lines to save screen space!):

        c:\test>555304 Thread 1 Thread 2 Thread 3 Thread 4 Via system: 1 2 3 4 5 6 7 8 9 Via system: 1 2 3 4 5 6 7 8 9 Via system: 1 2 3 4 5 6 7 8 9 Via system: 1 2 3 4 5 6 7 8 9 Via backticks 1 2 3 4 5 6 7 8 9 Via backticks 1 2 3 4 5 6 Via backticks 1 2 3 4 5 6 7 8 9 7 8 9 Via backticks 1 2 3 4 5 6 7 8 9 Via pipe 1 2 3 4 5 6 7 8 9 Via pipe 1 2 3 4 5 6 7 8 9 Via pipe 1 2 3 4 5 6 7 8 9 Via pipe 1 2 3 4 5 6 7 8 9

        As you can see, all four threads ran all 3 methods without problems. The "sorted" output is a little mixed up as you might expect with four threads all outputting to the screen concurrently, but no hangs or errors. I did this with AS811 and AS817 with the same results. I don't have AS813 installed, but I do not expect it to fail, but if it is, the answer is to upgrade.

        If upgrading doesn't fix your problem, then we'll need to look more closely at your environment to try and understand what is broken on your system/installation.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.