HTTPS transactions with LWP::UserAgent

mrguy123 has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks,
I have discovered a strange bug. I need to do a secure HTTP transaction (HTTPS) with the LWP::UserAgent. This is the code (https://www.guy.com is just an example):

#!/usr/bin/perl
use strict;
use LWP::UserAgent;
{
    ##The Perl Module that does the work
    my $ua = new LWP::UserAgent();

    ##The URL
    my $url = "https://www.guy.com/";

    ##Creating the request
    my $req = new HTTP::Request ('GET',$url);

    ##Getting the HTML page
    my $res = $ua->request($req);

    ##Printing the HTML page
    print $res->content."\n";
}
[download]

When I run this from the command line (./demo_https) it works (doesn't fetch anything, but doesn't crash). However, when I run it using the "perl command" (perl demo_https), I get that most hated of all errors: "Segmentation fault". I also get this error when using a private Perl path, and not /usr/bin/perl.
I have to admit that I work on a few different servers, and I only get this error on one of them, so a lot of you guys might not be able to reconstruct the bug. Also, I am using a relatively old version of LWP::UserAgent (because its what the customers have).
Has anyone seen this type of error before, and has any idea what I can do about it?
Thanks,
Guy (mrguy123)

Update: In answer to some of the (very useful) replies, using newer and better tools (like WWW::Mechanize) would be the ideal suggestion, but unfortunately this also has to work for customers that don't have these tools.

Comment on HTTPS transactions with LWP::UserAgent Download Code

Replies are listed 'Best First'.
Re: HTTPS transactions with LWP::UserAgent by shmem (Chancellor) on Dec 17, 2007 at 10:01 UTC
and I only get this error on one of them Your code pulls in Crypt::SSLeay which contains a shared object linked against libssl. Most likely there's a version mismatch or another inconsistency in the library chain. Has there perchance been an openssl update? --shmem _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply]
Re^2: HTTPS transactions with LWP::UserAgent by Herkum (Parson) on Dec 17, 2007 at 19:05 UTC
I have had version errors with openssl libraries before, I was using the most current ones available. Those were the ones that were not working, I had to go backwards to openssl 0.9.6 to get Crypt::SSLeay to function correctly. So it is worth noting that sometimes you have to go back a few versions to get something working.	[reply]
Re: HTTPS transactions with LWP::UserAgent by moritz (Cardinal) on Dec 17, 2007 at 09:48 UTC
First of all you can look into the bug trackers for LWP::UserAgent on CPAN and for perl, perhaps that error is already known. Second step is to check versions: do you run the latest maintainance release (5.8.8) of perl? If no, consider upgrading; there are reasons that 5.8.8 was released, and the most prominent is: bug fixes. Finally you can start a debugger (like gdb) or a memory debugger (like valgrind) and try to gather some information on where that segmentation fault occurs (perl core, or a library?), perhaps even build a perl with debugging symbols to find out more.	[reply]
Re^2: HTTPS transactions with LWP::UserAgent by mrguy123 (Hermit) on Dec 18, 2007 at 07:06 UTC
Hi, thank for your advance I do run the latest version of Perl (5.8.8), but for some reason the LWP::UserAgent is from 2001. Does that make sense, or is there a problem with the installation?	[reply]
Re: HTTPS transactions with LWP::UserAgent by snowhare (Friar) on Dec 17, 2007 at 11:24 UTC
Try running it as '/usr/bin/perl demo_https' instead. You might have a different install of perl in your PATH somewhere that is getting run instead when you run it shorthand as just 'perl'	[reply]