ithreads picks fight with LWP::Agent; everybody loses

BlairHippo has asked for the wisdom of the Perl Monks concerning the following question:

I'm writing what should be a simple web-load-testing script using ithreads, but I'm getting glibc seg faults of the "double free or corruption (fasttop)" variety (complete error output behind tag below). Though experimentation, I've determined that the troublesome bit is the line calling LWP::UserAgent -> request(). The test code isolating the problem is:

#!/usr/bin/perl -w

use HTTP::Request;
use LWP::UserAgent;
use threads;

my $TEST_URL = "http://www.blairhippo.com/";

for (my $i = 0; $i < 500; $i++) {
  my $thread = threads -> create ('execute_request', $TEST_URL);
  $thread -> detach();
}

print "If you can read this line, I have run to completion and failed 
+to fail.  Which makes my success a failure.  I'm so emo.\n";

sub execute_request {
  my $url = shift;
  my $request = HTTP::Request -> new ('GET', $url);
  my $agent = LWP::UserAgent -> new();
  my $response = $agent -> request($request);
}
[download]

(Note: The actual code doesn't detach the threads, but I get this error whether I'm detaching or joining them back in, so I'm detaching them for the sake of simplicity.)

I have very limited control over my environment; I'm using Perl 5.8.8 and that's unlikely to change, though I ought to be able to update individual modules. A workaround that doesn't stress my processor too badly (eg, nothing fork-based) would be more than welcome. The important functionality here is my ability to generate concurrent HTTP requests (and time how long they take).

My thanks in advance for any help or insight anybody is able to offer.

Edit: I just realized I didn't include a relevant detail. It doesn't seg fault right out of the gate; it runs a while (teasing me with the possibility that maybe this time I fixed it) before crashing. Dunno if that changes anybody's evaluation, but it's worth mentioning.

*** glibc detected *** /usr/bin/perl: double free or corruption (fastt
+op): 0x00002aaacc1b8db0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x2b38c83bb21d]
/lib64/libc.so.6(cfree+0x76)[0x2b38c83bcf76]
/usr/bin/perl(Perl_newCONSTSUB+0x144)[0x44e574]
/usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/IO/IO.so(boot_IO+0
+x34e)[0x2aaaaaaaccee]
/usr/bin/perl(Perl_pp_entersub+0x363)[0x486bb3]
/usr/bin/perl(Perl_runops_debug+0x11e)[0x468f8e]
/usr/bin/perl(Perl_call_sv+0x77d)[0x42492d]
/usr/bin/perl(Perl_call_list+0x30c)[0x424e7c]
/usr/bin/perl(Perl_newATTRSUB+0xe82)[0x458042]
/usr/bin/perl(Perl_utilize+0x24a)[0x45658a]
/usr/bin/perl(Perl_yyparse+0x247e)[0x449bde]
/usr/bin/perl[0x4b4de0]
/usr/bin/perl(Perl_pp_require+0xd49)[0x4b6fb9]
/usr/bin/perl(Perl_runops_debug+0x11e)[0x468f8e]
/usr/bin/perl(Perl_call_sv+0x77d)[0x42492d]
/usr/bin/perl(Perl_call_list+0x30c)[0x424e7c]
/usr/bin/perl(Perl_newATTRSUB+0xe82)[0x458042]
/usr/bin/perl(Perl_utilize+0x24a)[0x45658a]
/usr/bin/perl(Perl_yyparse+0x247e)[0x449bde]
/usr/bin/perl[0x4b4de0]
/usr/bin/perl(Perl_pp_require+0xd49)[0x4b6fb9]
/usr/bin/perl(Perl_runops_debug+0x11e)[0x468f8e]
/usr/bin/perl(Perl_call_sv+0x77d)[0x42492d]
/usr/bin/perl(Perl_call_list+0x30c)[0x424e7c]
/usr/bin/perl(Perl_newATTRSUB+0xe82)[0x458042]
/usr/bin/perl(Perl_utilize+0x24a)[0x45658a]
/usr/bin/perl(Perl_yyparse+0x247e)[0x449bde]
/usr/bin/perl[0x4b4de0]
/usr/bin/perl(Perl_pp_require+0xd49)[0x4b6fb9]
/usr/bin/perl(Perl_runops_debug+0x11e)[0x468f8e]
/usr/bin/perl(Perl_call_sv+0x77d)[0x42492d]
/usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/threads/threads.so
+[0x2b38c869c07e]
/lib64/libpthread.so.0[0x2b38c8138020]
/lib64/libc.so.6(clone+0x6d)[0x2b38c8411f8d]
======= Memory map: ========
00400000-00537000 r-xp 00000000 08:01 2424843                         
+   /usr/bin/perl
00736000-0073d000 rw-p 00136000 08:01 2424843                         
+   /usr/bin/perl
0073d000-023d4000 rw-p 0073d000 00:00 0                               
+   [heap]
40000000-40001000 ---p 40000000 00:00 0
40001000-40801000 rw-p 40001000 00:00 0
40801000-40802000 ---p 40801000 00:00 0
40802000-41002000 rw-p 40802000 00:00 0
41002000-41003000 ---p 41002000 00:00 0
41003000-41803000 rw-p 41003000 00:00 0
41803000-41804000 ---p 41803000 00:00 0
41804000-42004000 rw-p 41804000 00:00 0
42004000-42005000 ---p 42004000 00:00 0
42005000-42805000 rw-p 42005000 00:00 0
42805000-42806000 ---p 42805000 00:00 0
42806000-43006000 rw-p 42806000 00:00 0
43006000-43007000 ---p 43006000 00:00 0
43007000-43807000 rw-p 43007000 00:00 0
43807000-43808000 ---p 43807000 00:00 0
43808000-44008000 rw-p 43808000 00:00 0
44008000-44009000 ---p 44008000 00:00 0
44009000-44809000 rw-p 44009000 00:00 0
44809000-4480a000 ---p 44809000 00:00 0
4480a000-4500a000 rw-p 4480a000 00:00 0
4500a000-4500b000 ---p 4500a000 00:00 0
4500b000-4580b000 rw-p 4500b000 00:00 0
4580b000-4580c000 ---p 4580b000 00:00 0
4580c000-4600c000 rw-p 4580c000 00:00 0
4600c000-4600d000 ---p 4600c000 00:00 0
4600d000-4680d000 rw-p 4600d000 00:00 0
4680d000-4680e000 ---p 4680d000 00:00 0
4680e000-4700e000 rw-p 4680e000 00:00 0
4700e000-4700f000 ---p 4700e000 00:00 0
4700f000-4780f000 rw-p 4700f000 00:00 0
4780f000-47810000 ---p 4780f000 00:00 0
47810000-48010000 rw-p 47810000 00:00 0
48010000-48011000 ---p 48010000 00:00 0
48011000-48811000 rw-p 48011000 00:00 0
48811000-48812000 ---p 48811000 00:00 0
48812000-49012000 rw-p 48812000 00:00 0
49012000-49013000 ---p 49012000 00:00 0
49013000-49813000 rw-p 49013000 00:00 0
49813000-49814000 ---p 49813000 00:00 0
49814000-4a014000 rw-p 49814000 00:00 0
4a014000-4a015000 ---p 4a014000 00:00 0
4a015000-4a815000 rw-p 4a015000 00:00 0
4a815000-4a816000 ---p 4a815000 00:00 0
4a816000-4b016000 rw-p 4a816000 00:00 0
4b016000-4b017000 ---p 4b016000 00:00 0
4b017000-4b817000 rw-p 4b017000 00:00 0
4b817000-4b818000 ---p 4b817000 00:00 0
4b818000-4c018000 rw-p 4b818000 00:00 0
4e81d000-4e81e000 ---p 4e81d000 00:00 0
4e81e000-4f01e000 rw-p 4e81e000 00:00 0
51823000-51824000 ---p 51823000 00:00 0
51824000-52024000 rw-p 51824000 00:00 0
52825000-52826000 ---p 52825000 00:00 0
52826000-53026000 rw-p 52826000 00:00 0
2aaaaaaab000-2aaaaaaaf000 r-xp 00000000 08:01 2883850                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/IO/IO.so
2aaaaaaaf000-2aaaaacae000 ---p 00004000 08:01 2883850                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/IO/IO.so
2aaaaacae000-2aaaaacb0000 rw-p 00003000 08:01 2883850                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/IO/IO.so
2aaaaacc5000-2aaaaaccf000 r-xp 00000000 08:01 376857                  
+   /lib64/libnss_files-2.6.1.so
2aaaaaccf000-2aaaaaece000 ---p 0000a000 08:01 376857                  
+   /lib64/libnss_files-2.6.1.so
2aaaaaece000-2aaaaaed0000 rw-p 00009000 08:01 376857                  
+   /lib64/libnss_files-2.6.1.so
2aaaaaed0000-2aaaaaf05000 r--s 00000000 08:01 427755                  
+   /var/run/nscd/dbJBSLjV (deleted)
2aaaaaf06000-2aaaaaf10000 r-xp 00000000 08:01 2885090                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/List/Util/Util
+.so
2aaaaaf10000-2aaaab10f000 ---p 0000a000 08:01 2885090                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/List/Util/Util
+.so
2aaaab10f000-2aaaab110000 r--p 00009000 08:01 2885090                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/List/Util/Util
+.so
2aaaab110000-2aaaab111000 rw-p 0000a000 08:01 2885090                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/List/Util/Util
+.so
2aaaab111000-2aaaab116000 r-xp 00000000 08:01 2883847                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/File/Glob/Glob
+.so
2aaaab116000-2aaaab315000 ---p 00005000 08:01 2883847                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/File/Glob/Glob
+.so
2aaaab315000-2aaaab317000 rw-p 00004000 08:01 2883847                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/File/Glob/Glob
+.so
2aaaab317000-2aaaab31a000 r-xp 00000000 08:01 2883843                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/Fcntl/Fcntl.so
2aaaab31a000-2aaaab519000 ---p 00003000 08:01 2883843                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/Fcntl/Fcntl.so
2aaaab519000-2aaaab51b000 rw-p 00002000 08:01 2883843                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/Fcntl/Fcntl.so
2aaaab51b000-2aaaab528000 r-xp 00000000 08:01 376917                  
+   /lib64/libgcc_s.so.1
2aaaab528000-2aaaab727000 ---p 0000d000 08:01 376917                  
+   /lib64/libgcc_s.so.1
2aaaab727000-2aaaab729000 rw-p 0000c000 08:01 376917                  
+   /lib64/libgcc_s.so.1
2aaaac000000-2aaaad36a000 rw-p 2aaaac000000 00:00 0
2aaaad36a000-2aaab0000000 ---p 2aaaad36a000 00:00 0
2aaab0000000-2aaab1407000 rw-p 2aaab0000000 00:00 0
2aaab1407000-2aaab4000000 ---p 2aaab1407000 00:00 0
2aaab4000000-2aaab5248000 rw-p 2aaab4000000 00:00 0
2aaab5248000-2aaab8000000 ---p 2aaab5248000 00:00 0
2aaab8000000-2aaab91dd000 rw-p 2aaab8000000 00:00 0
2aaab91dd000-2aaabc000000 ---p 2aaab91dd000 00:00 0
2aaabc000000-2aaabc005000 r-xp 00000000 08:01 2883862                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/Socket/Socket.
+so
2aaabc005000-2aaabc204000 ---p 00005000 08:01 2883862                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/Socket/Socket.
+so
2aaabc204000-2aaabc206000 rw-p 00004000 08:01 2883862                 
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/Socket/Socket.
+so
2aaac0000000-2aaac13a8000 rw-p 2aaac0000000 00:00 0
2aaac13a8000-2aaac4000000 ---p 2aaac13a8000 00:00 0
2aaac4000000-2aaac5112000 rw-p 2aaac4000000 00:00 0
2aaac5112000-2aaac8000000 ---p 2aaac5112000 00:00 0
2aaac8000000-2aaac90fb000 rw-p 2aaac8000000 00:00 0
2aaac90fb000-2aaacc000000 ---p 2aaac90fb000 00:00 0
2aaacc000000-2aaacd2da000 rw-p 2aaacc000000 00:00 0
2aaacd2da000-2aaad0000000 ---p 2aaacd2da000 00:00 0
2aaad0000000-2aaad10c8000 rw-p 2aaad0000000 00:00 0
2aaad10c8000-2aaad4000000 ---p 2aaad10c8000 00:00 0
2aaad4000000-2aaad4027000 r-xp 00000000 08:01 378333                  
+   /usr/lib/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/Comp
+ress/Raw/Zlib/Zlib.so
2aaad4027000-2aaad4227000 ---p 00027000 08:01 378333                  
+   /usr/lib/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/Comp
+ress/Raw/Zlib/Zlib.so
2aaad4227000-2aaad4228000 r--p 00027000 08:01 378333                  
+   /usr/lib/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/Comp
+ress/Raw/Zlib/Zlib.so
2aaad4228000-2aaad4229000 rw-p 00028000 08:01 378333                  
+   /usr/lib/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/Comp
+ress/Raw/Zlib/Zlib.so
2aaad8000000-2aaad8f31000 rw-p 2aaad8000000 00:00 0
2aaad8f31000-2aaadc000000 ---p 2aaad8f31000 00:00 0
2aaadc000000-2aaadcfa7000 rw-p 2aaadc000000 00:00 0
2aaadcfa7000-2aaae0000000 ---p 2aaadcfa7000 00:00 0
2aaae0000000-2aaae000b000 r-xp 00000000 08:01 280473                  
+   /usr/lib/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/HT
+ML/Parser/Parser.so
2aaae000b000-2aaae020b000 ---p 0000b000 08:01 280473                  
+   /usr/lib/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/HT
+ML/Parser/Parser.so
2aaae020b000-2aaae020d000 rw-p 0000b000 08:01 280473                  
+   /usr/lib/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/HT
+ML/Parser/Parser.so
2aaae4000000-2aaae4ee0000 rw-p 2aaae4000000 00:00 0
2aaae4ee0000-2aaae8000000 ---p 2aaae4ee0000 00:00 0
2b38c7884000-2b38c78a0000 r-xp 00000000 08:01 378304                  
+   /lib64/ld-2.6.1.so
2b38c78a0000-2b38c78a1000 rw-p 2b38c78a0000 00:00 0
2b38c78a1000-2b38c78a2000 r--p 00000000 08:01 2851489                 
+   /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION
2b38c78a2000-2b38c78a9000 r--s 00000000 08:01 2854279                 
+   /usr/lib64/gconv/gconv-modules.cache
2b38c78a9000-2b38c78aa000 r--p 00000000 08:01 2853700                 
+   /usr/lib/locale/en_US.utf8/LC_MEASUREMENT
2b38c78aa000-2b38c78ab000 r--p 00000000 08:01 2853699                 
+   /usr/lib/locale/en_US.utf8/LC_TELEPHONE
2b38c78ab000-2b38c78ac000 r--p 00000000 08:01 2851488                 
+   /usr/lib/locale/en_US.utf8/LC_ADDRESS
2b38c78ac000-2b38c78ad000 r--p 00000000 08:01 2853814                 
+   /usr/lib/locale/en_US.utf8/LC_NAME
2b38c78ad000-2b38c78ae000 r--p 00000000 08:01 2853739                 
+   /usr/lib/locale/en_US.utf8/LC_PAPER
2b38c78ae000-2b38c78af000 r--p 00000000 08:01 2853921                 
+   /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES
2b38c78af000-2b38c78b0000 r--p 00000000 08:01 2851491                 
+   /usr/lib/locale/en_US.utf8/LC_MONETARY
2b38c78b6000-2b38c78b7000 rw-p 2b38c78b6000 00:00 0
2b38c78b7000-2b38c7997000 r--p 00000000 08:01 2853947                 
+   /usr/lib/locale/en_US.utf8/LC_COLLATE
2b38c7997000-2b38c7998000 r--p 00000000 08:01 2851492                 
+   /usr/lib/locale/en_US.utf8/LC_TIME
2b38c7998000-2b38c7999000 r--p 00000000 08:01 2853928                 
+   /usr/lib/locale/en_US.utf8/LC_NUMERIC
2b38c7999000-2b38c79d8000 r--p 00000000 08:01 2853946                 
+   /usr/lib/locale/en_US.utf8/LC_CTYPE
2b38c79d8000-2b38c79f9000 rw-p 2b38c79d8000 00:00 0
2b38c7a9f000-2b38c7aa1000 rw-p 0001b000 08:01 378304                  
+   /lib64/ld-2.6.1.so
2b38c7aa1000-2b38c7af3000 r-xp 00000000 08:01 376848                  
+   /lib64/libm-2.6.1.so
2b38c7af3000-2b38c7cf2000 ---p 00052000 08:01 376848                  
+   /lib64/libm-2.6.1.so
2b38c7cf2000-2b38c7cf4000 rw-p 00051000 08:01 376848                  
+   /lib64/libm-2.6.1.so
2b38c7cf4000-2b38c7cf6000 r-xp 00000000 08:01 376846                  
+   /lib64/libdl-2.6.1.so
2b38c7cf6000-2b38c7ef6000 ---p 00002000 08:01 376846                  
+   /lib64/libdl-2.6.1.so
2b38c7ef6000-2b38c7ef8000 rw-p 00002000 08:01 376846                  
+   /lib64/libdl-2.6.1.so
2b38c7ef8000-2b38c7f00000 r-xp 00000000 08:01 376844                  
+   /lib64/libcrypt-2.6.1.so
2b38c7f00000-2b38c8100000 ---p 00008000 08:01 376844                  
+   /lib64/libcrypt-2.6.1.so
2b38c8100000-2b38c8103000 rw-p 00008000 08:01 376844                  
+   /lib64/libcrypt-2.6.1.so
2b38c8103000-2b38c8132000 rw-p 2b38c8103000 00:00 0
2b38c8132000-2b38c8147000 r-xp 00000000 08:01 376866                  
+   /lib64/libpthread-2.6.1.so
2b38c8147000-2b38c8347000 ---p 00015000 08:01 376866                  
+   /lib64/libpthread-2.6.1.so
2b38c8347000-2b38c8349000 rw-p 00015000 08:01 376866                  
+   /lib64/libpthread-2.6.1.so
2b38c8349000-2b38c834d000 rw-p 2b38c8349000 00:00 0
2b38c834d000-2b38c8489000 r-xp 00000000 08:01 376840                  
+   /lib64/libc-2.6.1.so
2b38c8489000-2b38c8688000 ---p 0013c000 08:01 376840                  
+   /lib64/libc-2.6.1.so
2b38c8688000-2b38c868b000 r--p 0013b000 08:01 376840                  
+   /lib64/libc-2.6.1.so
2b38c868b000-2b38c868d000 rw-p 0013e000 08:01 376840                  
+   /lib64/libc-2.6.1.so
2b38c868d000-2b38c8694000 rw-p 2b38c868d000 00:00 0
2b38c8694000-2b38c869e000 r-xp 00000000 08:01 196730                  
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/threads/thread
+s.so
2b38c869e000-2b38c889d000 ---p 0000a000 08:01 196730                  
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/threads/thread
+s.so
2b38c889d000-2b38c889e000 r--p 00009000 08:01 196730                  
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/threads/thread
+s.so
2b38c889e000-2b38c889f000 rw-p 0000a000 08:01 196730                  
+   /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/auto/threads/thread
+s.so
7fffe3211000-7fffe3226000 rw-p 7fffe3211000 00:00 0                   
+   [stack]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0               
+   [vdso]
Aborted
[download]

Comment on ithreads picks fight with LWP::Agent; everybody loses Select or Download Code

Replies are listed 'Best First'.
Re: ithreads picks fight with LWP::Agent; everybody loses by perrin (Chancellor) on Feb 27, 2008 at 00:02 UTC
You seem to be under the impression that forking will use more resources than threads. Usually just the opposite is true with perl threads on a unix-like platform. I strongly recommend you try forking instead.	[reply]
Re^2: ithreads picks fight with LWP::Agent; everybody loses by BlairHippo (Initiate) on Feb 27, 2008 at 07:04 UTC
Uh-oh. I'm actually doing this the other way around; the original code went crazy with forks (I'm trying to get a LOT of HTTP requests running concurrently) and quite simply brought the server to its knees. Thanks for the warning; it'll keep me from committing seppuku if it turns out I'm wasting all this effort barking up another wrong tree.	[reply]
Re: ithreads picks fight with LWP::Agent; everybody loses by ysth (Canon) on Feb 26, 2008 at 23:44 UTC
Have you tried upgrading to the latest threads from CPAN? (Changes since 5.8.8's version 1.07.) -- Online Fortune Cookie Search	[reply]
Re^2: ithreads picks fight with LWP::Agent; everybody loses by BlairHippo (Initiate) on Feb 27, 2008 at 06:59 UTC
Yup, we're using 1.69.	[reply]
Re: ithreads picks fight with LWP::Agent; everybody loses (Works for me!) by BrowserUk (Patriarch) on Feb 27, 2008 at 00:39 UTC
It works for me as posted. I added a couple of trace lines to convince myself: `[ 0:29:23.03]C:\test>670431.pl thread 1 started thread 2 started thread 3 started thread 4 started thread 5 started thread 6 started thread 7 started thread 9 started thread 8 started thread 10 started [SNIP] thread 474 ended thread 244 ended thread 239 ended thread 496 started thread 497 started thread 498 started thread 499 started If you can read this line, I have run to completion and failed to fail +. Which makes my success a failure. I'm so emo.` [download] As you are detaching the threads, they weren't all able to complete before the main thread ended. So rather than all that stuff above, how about posting some useful information like the versions of the modules you are using. Particularly threads Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^2: ithreads picks fight with LWP::Agent; everybody loses (Works for me!) by Anonymous Monk on Feb 27, 2008 at 03:55 UTC
He is using threads 1.69	[reply]
Re^3: ithreads picks fight with LWP::Agent; everybody loses (Works for me!) by BrowserUk (Patriarch) on Feb 27, 2008 at 04:00 UTC
Perhaps he could try reverting to v1.67 (the version I have that works, albeit on a win32 build). He might also try the following and see if it completes clean (and post the output if not): #!/usr/bin/perl -w use POSIX qw[ _exit ]; use Time::HiRes qw[ usleep ]; use threads ( stack_size => 4096 ); use threads::shared; $\|++; use HTTP::Request; use LWP::UserAgent; my $started :shared = 0; my $ended :shared = 0; my $monitor = async { printf "\rStarted: $started Ended: $ended\t" and uleep 100 while $ended < 500; print "\nAll done"; }; my $TEST_URL = "http://www.blairhippo.com/"; threads->create( 'execute_request', $TEST_URL )->detach for 1 ..500; $monitor->join; _exit( 0 ); sub execute_request { { lock $started; ++$started; } my $url = shift; my $request = HTTP::Request -> new ('HEAD', $url); my $agent = LWP::UserAgent -> new(); my $response = $agent -> request($request); { lock $ended; ++$ended; } } [download] Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^4: ithreads picks fight with LWP::Agent; everybody loses (Works for me!) by BlairHippo (Initiate) on Feb 27, 2008 at 07:23 UTC
Re^5: ithreads picks fight with LWP::Agent; everybody loses (Works for me!) by BrowserUk (Patriarch) on Feb 27, 2008 at 07:52 UTC
Re: ithreads picks fight with LWP::Agent; everybody loses by gferguson (Acolyte) on Feb 27, 2008 at 00:04 UTC
I did some stuff using threading, but kept getting a nervous tic when I would see the warning about threading and DBI while compiling things. I never had problems but it was such a dire message.... So, as an experiment I tried using LWP::Parallel for a project. That worked nicely and allowed me to throw 8 requests/second at my target. If I'd have had more bandwidth on the hosting machine and the network I could probably have gone higher, but 8 was the point where I saw no increase in throughput. I never dived into the code to learn more about the why-fors on the whole threading thing. I never had problems with it, but wanted to point out there are alternatives.	[reply]
Re^2: ithreads picks fight with LWP::Agent; everybody loses by BlairHippo (Initiate) on Feb 27, 2008 at 07:07 UTC
Ooh! I didn't know LWP::Parallel existed. Thanks, this could be exactly what I need!	[reply]
Re: ithreads picks fight with LWP::Agent; everybody loses by jepri (Parson) on Feb 27, 2008 at 03:32 UTC
I notice you're using a 64-bit install. How does it run on a 32-bit machine? Low level stuff like threads is notorius for going sproing on 64 bit machines. ___________________ Jeremy Bots of the disorder.	[reply]
Re^2: ithreads picks fight with LWP::Agent; everybody loses by BlairHippo (Initiate) on Feb 27, 2008 at 07:27 UTC
That I do not know; I haven't had the opportunity to run it on a 32-bit machine. I'll see if we have one handy, and thanks for the recommendation. I was unaware that 64-bit installs have that rep. This is obviously a hole in my knowledge that needs filling. Are there any resources in particular you'd recommend discussing what to avoid in Perl on a 64-bit machine? When the code goes "sproing!", so does my performance review.	[reply]
Re^3: ithreads picks fight with LWP::Agent; everybody loses by jepri (Parson) on Feb 27, 2008 at 10:24 UTC
I don't have any experience in 64 bit perl beyond "it worked for me". But then, I wasn't doing something suicidal like using threads. My statement was more based on having watched windows and "Linux" flop around on the x86 64-bit beach as they attempted to drag themselves out of the 32-bit sea. So to speak. Linux and Perl have been 64-bit clean since before x86-64 existed, but there's no guarantees that other code on the system is clean and won't interfere in weird ways. Are there any resources in particular you'd recommend discussing what to avoid in Perl on a 64-bit machine? Don't use source filters. Also, don't use threads. Keep away from extensions written in C, since most C programmers still can't understand why they shouldn't store pointers in INTs. More generally, (system) threads are a bad solution for most problems. If you want to download a lot of files, for instance, you'll be better served using asynchronous calls than threads. For starters, your programs won't crash. Edit: use something like this instead: forks ___________________Jeremy_______Bots of the disorder.	[reply]

Back to Seekers of Perl Wisdom