So claiming before that it most definitely was in threads->detach is simply not true. At least not by itself.

I have severe criticism on the implementation of threads in Perl. You will find some of it at Things you need to know before programming Perl ithreads. And I too have been frustrated at the seemingly random way of crashing that threaded Perl programs sometimes have. But in the end, there's a reason for it: by using threads, it just becomes an order of magnitude harder to trace and fix.

When you're debugging threaded programs, you need to develop a brain that can run several processes simultaneously, which is pretty hard if you're used to just needing to follow a single process' logic. At every point in your code, you need to make sure that there can not be another thread intervening.

Some (core) libraries may not be there yet. libnet may be one of them.

One final note: since you seem to be using Activestate's Perl (I'm assuming here, since you said you downloaded a binary for Win32), it may be wise to also alert ActiveState of your problem as well: most of the Win32 Perl expertise is located at ActiveState, and interaction with p5p has recently become what I would call "suboptimal".

One more final note: don't count on it being fixed for 5.8.3. Nicholas Clark has indicated that only stable bug fixes will be allowed in at this point. Without even an indication of the cause of the problem, I don't think a bug fix will be stable enough to be deemed ok by Nick. On the bright side: I wouldn't be surprised if the problem is in libnet, which would imply a version of CPAN (or ActiveState's ppm repository) could become available quickly after the problem has been found and fixed.

Liz

liz, please see my reply above to castaway, so it breaks without socket. (You did made me thought that it might be the combination of socket and thread caused problems, and that's why I intentionally came up with something really really simple.)

But as I said in our conversation, I do appreciate your comment, and it has such a rich content.

My thought sometime jumps too quick (as everyone knows me said), and I really like your style: calm and with lots of thought.

Thanks again, for not only this reply, but also your reply to my first notes on .2 threads.

most of the Win32 Perl expertise is located at ActiveState, and interaction with p5p has recently become what I would call "suboptimal".

Thats an interesting comment that Id like to hear expanded upon if you feel like doing so....



---
demerphq



First they ignore you, then they laugh at you, then they fight you, then you win.
-- Gandhi

Can you cut the code down to something that shows that its actually 'detach' thats breaking it, and why? (just removing/adding it might have some other effects).

Maybe you could also find out where memory is leaking.. One of the nice things I noticed after upgrading to 5.8.2 was that my script (the im2 script, actually) stopped leaking memory.. in 5.8.0 and 5.8.1 it was getting to around 150mb before leveling off, now in 5.8.2 it can run for days at a time and only peak around 25mb.

Without trying to find where the bugs are exactly, I doubt you will get much response from p5p, since they probably dont want to dig through your code, and wont get much out of a 'detach produces core dumps' error..

C. still struggling to figure out if blessed objects 'work' .. currently they seem to be working, darn it..

"Can you cut the code down to something that shows that its actually 'detach' thats breaking it"

Okay, that's easy. It only took me 1 minute to come up with this code, and break (core dump) it: (I repeated 3 times to make sure it is really broken)

use strict;
use warnings;
use threads;

for (my $i = 1; $i < 1000; $i ++) {
    threads->create(\&abc)->detach();
}

sub abc {
}
[download]

Update:

It was a waste to create 1000 threads, and side tracks people's focus. This is much more simple and clear:

use strict;
use warnings;
use threads;

for (my $i = 1; $i < 3; $i ++) {
    threads->create(\&abc)->detach();
}
<>;

sub abc {
}
[download]

Update 2:

The testing is done on win32, using .2. This was mentioned in the first stream of notes Perl 5.8.2 thread is much worse on .2 threads. Any way, I should have repeated this here earlier.

Could you try it with this change (in pseudo diff format):

 sub abc {
+undef
 }
[download]

A bug (new to 5.8.2) causes problems with empty subroutines. Put the undef in there, and see if that helps. I've had to update several of my modules to get around this problem for threaded 5.8.2.

I should have thought about this earlier. The whole thing sort of left my mind already... ;-) Good thing I document my changes... ;-)

BTW, this should already be fixed for 5.8.3.

Liz

I can partly confirm the results with my perl 5.8.2 under Windows 2000 - some times, the snippet crashes with an access violation, but sometimes it dosen't. I could not safely reproduce the bug all times. This looks to me like a weird bug in Perl to me, some uninitialized pointer somewhere.

Update: In my search for the cause, I found the following behaviour - the segfault appears within a spawned thread, as the following script exits even as the segfault messagebox is still on screen:

use strict;
use warnings;
use threads;

for (1..2) {
    threads->create(sub { undef })->detach();

};

sleep 1;
[download]

Summary of my perl5 (revision 5 version 8 subversion 2) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define usemultip
+licity=def
ine
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32 -D_CON
+SOLE -DNO_
STRICT -DHAVE_DES_FCRYPT  -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS 
+-DUSE_PERL
IO -DPERL_MSVCRT_READFIX',
    optimize='-MD -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64
+', lseeksi
ze=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -release  -libpath:"d:\
+perl\5.8.2
\lib\CORE"  -machine:x86'
    libpth=C:\PROGRA~1\MICROS~3\VC98\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
+  comdlg32
.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uui
+d.lib wsoc
k32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.
+lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool
+.lib  comd
lg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib
+ uuid.lib
wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msv
+crt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release  -l
+ibpath:"d:
\perl\5.8.2\lib\CORE"  -machine:x86'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL
+_IMPLICIT_
CONTEXT PERL_IMPLICIT_SYS
  Built under MSWin32
  Compiled at Nov  6 2003 12:48:55
  @INC:
    D:/perl/5.8.2/lib
    D:/perl/site/5.8.2/lib
    .
[download]

perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web
[download]

No core dump using that code on my perl 5.8.2:

This is perl, v5.8.2 built for i686-linux-thread-multi

Copyright 1987-2003, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'.  If you have access to the
Internet, point your browser at http://www.perl.com/, the Perl Home Page.

C.

If invoke perl interactively like so, no memory access violations.

E:\>perl
use strict;
use warnings;
use threads;

for (my $i = 1; $i < 3; $i ++) {
    threads->create(\&abc)->detach();
}
<>;

sub abc {
}
__END__


E:\>perl -V
Summary of my perl5 (revision 5 version 8 subversion 2) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define usemultip
+licity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32 -D_CON
+SOLE -DNO_STRICT -DHAVE_DES_FCRYPT   -DPERL_IMPLIC
IT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64
+', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -release  -libpath:"G:\
+perls\5.8.2\lib\CORE"  -machine:x86'
    libpth=\lib \site\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
+  comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  ver
+sion.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool
+.lib  comdlg32.lib advapi32.lib shell32.lib ole32.
lib oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib 
+ version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release  -l
+ibpath:"G:\perls\5.8.2\lib\CORE"  -machine:x86'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL
+_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
  Built under MSWin32
  Compiled at Nov  5 2003 21:13:21
  %ENV:
    PERL5LIB="G:\perls\5.8.2\site\lib;"
  @INC:
    G:\perls\5.8.2\site\lib
    G:/perls/5.8.2/lib
    .

E:\>
[download]

update: ok, if I save it to a file, it happens, d'oh. The bugs appears in threads.dll according to mah debugger (Unhandled exception in perl.exe(THREADS.DLL): 0XC0000005 ). And apparently, line 152 in threads.xs is the culprit (        PerlMemShared_free(thread);)

update: I should note that both the threads cause this violation, but that this is only apparent in the debug version (in the non-debug version it's death after the 1st), and that like corion, I can't reliably reproduce this (sometimes I have no problems running the snippet, sometimes I do).

It appears that the interactive invocation can cause the error also.

Read more... (3 kB)

MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"

I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).

** The third rule of perl club is a statement of fact: pod is sexy.

under cygwin, which basicaly is win32 with a *nix layer this code core dump

# echo|perl thisscript
use strict;
use warnings;
use threads;

for (my $i = 1; $i < 3; $i ++) {
    threads->create(\&abc)->detach();
}
<>;

sub abc {
}
[download]

while this one is OK

#perl
use strict;
use warnings;
use threads;
for (my $i = 1; $i < 3; $i ++) {
    threads->create(\&abc)->detach();
}
sleep 1;
sub abc {
}
[download]

... I've tried to mix case of empty sub and processing the <>.
--
dominix

Now, I could envision a temporary hack which would use Thread::Exit and that would steal threads->detach to do nothing except marking it as "detached", and stealing threads->new to join() all the threads marked as "detached", so that at least the memory would be freed of the finished threads. I could also put this into a seperate module, e.g. Thread::Detach, which would disable itself for future versions of Perl where this problem would fixed on Win32.

Would that help?

Liz

It rather sounds like they should be using Thread Local Storage*, instead of Global Memory* or Heap memory*. The problem is, the design of perl's memory management doesn't make it easy to use multiple memory allocation strategies for different portions of the interpreter.

^*links won't work correctly unless your browser pretends to be (or is) IE or netscape.

---

Examine what is said, not who speaks.

"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Hooray!

For me with ActiveState 5.8.2 on Windows 2000 the dropped scalars and memory leaks I was having with code I was working on is gone and my threads are running much better. I will look at the code you have but I guess some of the improvements I am having broke other things, but that has been the caveat with threads, ithreads particularly. This is an experiment and it may change dramatacially with each release. So is it a bug or is just the way ithreads are going. For me I am enjoying the change (at the moment).

As a general question since ithreads are an experiment that is wanted to be included is there any area within the Perl community to feedback on this subsystem specifically???

As a general question since ithreads are an experiment that is wanted to be included is there any area within the Perl community to feedback on this subsystem specifically???

Check out http://lists.perl.org for the iThreads mailing list.

Liz