Multi- Processing / threading under Win32

demerphq has asked for the wisdom of the Perl Monks concerning the following question:

I've got some questions about multi-processing and threading and stuff under Win32. My questions are primarily aimed at 5.6.1 and not the new threading stuff in 5.8.x. In short im curious what the differences are of using fork() and using system(1,...) on a multi CPU Win32 box. My understanding is that fork() on win32 is implemented using threading whereas system(1,...) will spawn a full new process. So does this mean for instance that if a forked process dies it takes down the whole thing? As far as I can tell under system(1,...) it won't. Likewise various hazy questions regarding processor affinity and the like come to mind.

Anyway, part of this is motivated because of a buffer overrun bug with system(1,STRING) where string can be no more than 257 characters or perl segfaults. Can i replace my use of this command with forking safely? Bear in mind each process needs DB connections etc so if forking has problem there i cant use it.

Anyway, here is an example of system(1,STRING) and potentially of the buffer overrun bug:

use strict;
use warnings;
use POSIX qw(:sys_wait_h);

my $segfault=0; # set this to 188 to make perl segfault under 5.6.1 

print $$,$/;
$|++;
my @pid;
foreach my $i (0..3) {
    my $cmd='perl -le "print qq(hello world!); sleep '
           .(10+$i*2)
           .(';' x $segfault )
           .'; print qq(foo); " >NUL 2>&1';
    print length($cmd),$/;
    $pid[$i]=system(1,$cmd);
}
my @wait=(0) x 4;
my $c=0;
while (grep !$_,@wait) {
    print(qq(running... @{[$c++]} [@wait]\r));
    foreach my $i (0..3) {
        $wait[$i]||=waitpid($pid[$i],&WNOHANG);
    }
    sleep(1);
}
print(qq(Finished:  @{[$c++]} [@wait]\r));
[download]

Thanks in advance for your help and time.

---
demerphq

_{First they ignore you, then they laugh at you, then they fight you, then you win.

-- Gandhi}

_{• Update:

Added the missing use() that confused browserUk}

Comment on Multi- Processing / threading under Win32 Download Code

Replies are listed 'Best First'.
Re: Multi- Processing / threading under Win32 by BrowserUk (Patriarch) on Sep 01, 2004 at 17:42 UTC
I can't reproduce your crash? ( I substituted 0 for &WNOHANG, whatever that is?) `C:\perl561\bin>perl5.6.1 p:387607.pl 644 258 258 258 258 Undefined subroutine &main::WNOHANG called at p:387607.pl line 22. C:\perl561\bin>perl5.6.1 p:387607.pl 700 258 258 258 258 Finished: 1 [1148 1380 1444 1196] C:\perl561\bin>perl5.6.1 p:387607.pl 840 327 327 327 327 Finished: 1 [1976 1708 584 1916]` [download] Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "Think for yourself!" - Abigail "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon	[reply] [d/l]
Re: Multi- Processing / threading under Win32 by ikegami (Patriarch) on Sep 02, 2004 at 00:48 UTC
For me, `Crashes using ActivePerl v5.6.1 build 633 on Win2k (single CPU)` `Works using ActivePerl v5.6.1 build 635 on WinXP (single CPU)` For fun, `Works using ActivePerl v5.8.0 build 806 on WinXP (single CPU)`	[reply] [d/l] [select]
Re: Multi- Processing / threading under Win32 by BrowserUk (Patriarch) on Sep 01, 2004 at 19:05 UTC
Even with your update, I still can't reproduce your crash? C:\perl561\bin>perl5.6.1 -V Summary of my perl5 (revision 5 version 6 subversion 1) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef usethreads=undef use5005threads=undef useithreads=define usemultip +licity=define useperlio=undef d_sfio=undef uselargefiles=undef usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler: cc='cl', ccflags ='-nologo -O1 -MD -Zi -DNDEBUG -DWIN32 -D_CONSOLE + -DNO_STRICT -DHAVE_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLIC +IT_SYS -DPERL_MSVCRT_READFIX', optimize='-O1 -MD -Zi -DNDEBUG', cppflags='-DWIN32' ccversion='', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', + lseeksize=4 alignbytes=8, usemymalloc=n, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -l +ibpath:"c:\perl561\lib\CORE" -machine:x86' libpth="c:\cl\lib" "C:\HLA\hlalib" "" "c:\hla\hlalib" "c:\perl561\ +\lib\CORE" libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib + comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netap +i32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.li +b odbccp32.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool +.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib n +etapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc3 +2.lib odbccp32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl56.lib Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt: +ref,icf -libpath:"c:\perl561\lib\CORE" -machine:x86' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY USE_ITHREADS PERL_IMPLICIT_CONTEX +T PERL_IMPLICIT_SYS Locally applied patches: ActivePerl Build 638 Built under MSWin32 Compiled at Apr 13 2004 19:24:21 %ENV: PERLDB_OPTS=""Nonstop frame=2"" @INC: C:/perl561/lib C:/perl561/site/lib . C:\perl561\bin>type p:387607.pl use strict; use warnings; use POSIX qw(:sys_wait_h); my $segfault=257; # set this to 188 to make perl segfault under 5.6.1 print $$,$/; $\|++; my @pid; foreach my $i (0..3) { my $cmd='perl -le "print qq(hello world!); sleep ' .(10+$i*2) .(';' x $segfault ) .'; print qq(foo); " >NUL 2>&1'; print length($cmd),$/; $pid[$i]=system(1,$cmd); } my @wait=(0) x 4; my $c=0; while (grep !$_,@wait) { print(qq(running... @{[$c++]} [@wait]\r)); foreach my $i (0..3) { $wait[$i]\|\|=waitpid($pid[$i], &WNOHANG); } sleep(1); } print(qq(Finished: @{[$c++]} [@wait]\r)); C:\perl561\bin>perl5.6.1 p:387607.pl 1328 327 327 327 327 Finished: 18 [1364 928 1020 736] [download] Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "Think for yourself!" - Abigail "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon	[reply] [d/l]
Re^2: Multi- Processing / threading under Win32 by demerphq (Chancellor) on Sep 01, 2004 at 19:08 UTC
Well, thats great to know, now the thing is what version of perl are you using exactly? Is it an AS build? If so which? BTW, the ouput is a little more interesting in pasted form if you change the \r to an \n. :-) Update: Whoo-hoo. So AS 638 fixes this bug? Time for an upgrade I think. :-) Thanks BrowserUk!! --- demerphq _{First they ignore you, then they laugh at you, then they fight you, then you win. -- Gandhi}	[reply]