in reply to Re^6: Threads calling LWP causes exception
in thread Threads calling LWP causes exception

I setup a Windows 2003 Server without MS VS .NET 2003 installed on it. The Perl code works fine.

Strange, but not entirely surprising. AS is built with VC++ v6, and I continue to build anything intended for use with Perl using the same compiler. Theoretically unnecessary, but maybe my instincts are good :)

BTW, how can I get rid of the "Scalars leaked"?

Change your LetsRock sub to call threads->create directly instead of via your (rather useless. Sorry:), Thread wrapper and the problem goes away.

sub LetsRock { my (@kids); for (my $x=0; $x < 10; $x++) { my ($kid) = threads->create( \&ThreadFunc, 'https://www.wellsf +argo.com'); if (defined($kid)) { push(@kids, $kid); } sleep(1); } WaitForThreads(@kids); }

Leaked scalars generally seem to be related to the use of variables passed to the thread through closure (as of 5.8.6 at least). Using the asynch() function seems particularly prone to this. Quite why this

sub Thread { my ($function, @params) = @_; my ($thread); if (defined(&{$function})) { $thread = threads->create($function, @params); } return($thread); }

causes a leaked scalar I'm not sure, maybe the defined(&{$function}), but I fail to see the benefit of that piece of indirection anyway? In fact, the whole sub seems pretty redundant to me.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^8: Threads calling LWP causes exception
by mojoshaneman (Novice) on Mar 10, 2006 at 20:59 UTC
    Thanks for your help.
      Would you mind running this code with a command line of "https://www.wellsfargo.com" ? I have a feeling that Crypt-SSLeay is not thread safe.
      package perl_module1; use strict; use v5.8; use HTTP::Cookies; use LWP::UserAgent; use HTTP::Request::Common; use URI::Escape; use vars qw(@ISA @EXPORT); use Exporter (); our @ISA = qw(Exporter); our @EXPORT = qw( GetIt ); our $DEBUG = 0; sub GetIt { my ($url) = @_; my ($ua) = LWP::UserAgent->new(keep_alive => 1); $ua->timeout(60); # allow redirects for POST push @{ $ua->requests_redirectable }, 'POST'; # set the cookie my ($cookie) = './ac_cookie.txt' . $$; $ua->cookie_jar(HTTP::Cookies->new('file' => $cookie)); my ($response) = $ua->get($url); if ($response->is_success) { my ($page) = $response->content; print( $page ); } else { printf "Failed with %s\n", $response->message; } } return 1 if caller; package main; use strict; use warnings; use v5.8; use threads; use threads::shared; print("Starting LetsRock() \n"); ThreadFunc( $ARGV[ 0 ] ); LetsRock( $ARGV[ 0 ] ); print("LetsRock() ended \n"); exit(0); sub LetsRock { my $url = shift; my (@kids); for (my $x=0; $x < 5; $x++) { my ($kid) = threads->create( \&ThreadFunc, $url ); if (defined($kid)) { push(@kids, $kid); } sleep(1); } printf "Waiting for %s\n", $_->tid, $_->join for @kids; } sub ThreadFunc { return perl_module1::GetIt( $_[0] ); } __END__

        You may be happy to know that I finally reproduced your problem here using the above code, and I've reduce the failure case to this:

        package main; use strict; use warnings; use v5.8; use POSIX qw[ _exit ]; use threads; sub GetIt2 { require LWP::Simple; return LWP::Simple::get( $_[0] ); } #print GetIt2( $ARGV[ 0 ] ); #print threads->create( \&GetIt2, $ARGV[ 0 ] )->join; #_exit(0);

        This seems to be a bad interaction during cleanup between libeay32.dll and thread.pm.

        The keys to the failure seem to be:

        1. Using LWP for fetch a https url.
        2. Calling the thread func directly from the main thread before using as a thread procedure.
        3. Retrieving the return values (joining) the thread.
        4. Allowing the perl cleanup code to run.

        It will require somebody with greater internals skills to resolve that further, so you should raise a perlbug. The code above should suffice as a testcase.

        I've two workarounds for you.

        1. Don't both call your ThreadProc() as a synchronous function and use it as a thread procedure.

          The failure does not seem to occur unless you do that.

        2. If you really cannot avoid that situation in your code, the only call it from the main thread after you've started the threads.

          The error does not seem to occur then either.

        3. And if you really, really cannot not avoid the problem in either of these two ways, then you might consider using POSIX::_exit() to terminate your code.

          This is not a good solution as it effectively bypasses Perl's cleanup, which means open files don't get closed properly, destructors don't get called etc. If your code relies on any of the cleanup events for proper function, it could give you problems.

          That said, if you ensure that all your files are closed and data is saved before you call POSIX::_exit(), it will avoid the exception, so it worth considering as a last resort until a proper solution is found and implemented.

          Disclaimer: It's at your own risk and I didn't tell you about it :)


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.