Segmentation fault using LWP and threads

MarkSWarren has asked for the wisdom of the Perl Monks concerning the following question:

Here is a simple perl program using LWP::UserAgent and threads, that succeeds with http, but segfaults with https. Tried on RH9 and Fedora FC2, both with up to date patches. Can anyone tell me what's happening here?

Try running this code as

    perl program.pl 'http://www.yahoo.com'

and it reports success. But, if you run

    perl program.pl 'https://www.yahoo.com'

we get a segfault. Here's the smallest program we could make that shows this problem. Thanks in advance...

use strict;
use threads;
use LWP;

my $uri;
if (@ARGV > 0) {
    $uri = shift;
}

my $ua = LWP::UserAgent->new() or die "Can't create ua\n";

if (defined $uri) {
    print "uri: $uri\n";
    my $r = $ua->get($uri);
    if (not $r->is_success()) {
        print "GET error on <$uri>...\n" . $r->error_as_HTML() . "\n";
        exit(1);
    }
}

my $t = threads->new(\&threadedRequestThread)
  or die "Can't create thread\n";

print "Joining thread\n";
my @ret = $t->join();
print "Thread returned (@ret)\n";
exit(0);

# Actual Thread function
sub threadedRequestThread {
    print "Inside threadedRequestThread\n";
    return "A list of words";
}
[download]

Comment on Segmentation fault using LWP and threads Download Code

Replies are listed 'Best First'.
Re: Segmentation fault using LWP and threads by Corion (Patriarch) on Nov 12, 2004 at 15:12 UTC
You don't tell us which SSL library you are using, but did you make sure that it is threadsafe and compiled for threading ? As LWP dynamically loads stuff, you could maybe test the following program to further narrow it down: `use strict; use Net::SSLeay; # or whatever other SSL implementation you use use threads; my $t = threads->new(\&threadedRequestThread) or die "Can't create thread\n"; print "Joining thread\n"; my @ret = $t->join(); print "Thread returned (@ret)\n"; exit(0); # Actual Thread function sub threadedRequestThread { print "Inside threadedRequestThread\n"; return "A list of words"; }` [download] or, failing that program, you might need to call some routines in Net::SSLeay to initialize it.	[reply] [d/l]
Re^2: Segmentation fault using LWP and threads by Ray Smith (Beadle) on Nov 12, 2004 at 21:38 UTC
Our problem seams to appear when we are using all three: LWP, HTTPS, and threads. I tried using Net::SSLeay that you use - still segmentation fault. Note that we run under "perl -d" we don't get the segmentation fault but rather the join never returns. In a larger case which also gives a segmentation fault, we get "Bad signature in parser state object at 88de850 during global destruction." when we try to join the completed thread. We are using perl v5.8.0 built for i386-linux-thread-multi I thank you for your help so far. Ray Smith @ raysmith@alum.mit.edu	[reply]
Re^2: Segmentation fault using LWP and threads by MarkSWarren (Initiate) on Nov 12, 2004 at 22:45 UTC
On the FC2 system, we have: Perl 5.8.3 Net::SSLeay 1.25 openssl-0.9.7a-35 Also, I ran code you suggested (explicitly use Net::SSLeay, and them I did a call to get_https('www.yahoo.com', 443, '/') just to make sure SSL was fully initialized). This ran fine, no errors. So we're still stuck.	[reply]
Re: Segmentation fault using LWP and threads by Roger (Parson) on Nov 12, 2004 at 15:13 UTC
How about reducing the number of variables during your investigation by fetching the https link without thread first, if that works, then it's probably that the SSL library is not thread safe.	[reply]
Re^2: Segmentation fault using LWP and threads by MarkSWarren (Initiate) on Nov 12, 2004 at 22:50 UTC
LWP by itself has no problems. The segfault only occurs when threads are in the picture. How can I check to see if my SSL lib is built thread safe? I assume I need to check both Net::SSLeay and openssl ?	[reply]
Re: Segmentation fault using LWP and threads by BrowserUk (Patriarch) on Nov 12, 2004 at 17:37 UTC
FYI. I don't get any errors or segfaults from this on Win32/5.8.4/AS810. Which version of Perl are you using? Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "Think for yourself!" - Abigail "Time is a poor substitute for thought"--theorbtwo "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon	[reply]
Re^2: Segmentation fault using LWP and threads by Ray Smith (Beadle) on Nov 12, 2004 at 21:57 UTC
Question are you accessing HTTPS. We are and it appears that that requires Net::SSLeay; which does not appear to install on our Win32 system. We are using perl 5.8.0 under RedHat. Thanks for your suggestions.	[reply]
Re^3: Segmentation fault using LWP and threads by BrowserUk (Patriarch) on Nov 12, 2004 at 22:10 UTC
are you accessing HTTPS? Yes. I tried both the yahoo urls you posted. `[22:06:16.45] P:\test>perl -MNET::SSLeay -MLWP -le"print length LWP::U +serAgent->new()->get( 'https://www.yahoo.com')->content" 31091` [download] We are using perl 5.8.0 under RedHat. The OS doesn't matter in this case. iThreads were quite badly broken on all platforms prior to 5.8.3. Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "Think for yourself!" - Abigail "Time is a poor substitute for thought"--theorbtwo "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon	[reply] [d/l]
Re^4: Segmentation fault using LWP and threads by MarkSWarren (Initiate) on Nov 16, 2004 at 00:46 UTC