LWP and Oracle Portal HTTPS site problem

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I am using the following script, but it failed with "500 read failed" with the long URL get request:

#!/usr/bin/perl -w
use strict;
use HTTP::Cookies;
use LWP::UserAgent;
use URI::URL;
my $ua = LWP::UserAgent->new;
$ua->requests_redirectable ( ['GET', 'HEAD', 'POST'] );
my $cookie_jar = HTTP::Cookies->new; 
$ua->cookie_jar ($cookie_jar);
$ua->agent ("Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"); # p
+retends to be IE6
my $url = url 'https://advertise.latimes.com/';
my $res = $ua->get($url);
print $res->is_success,"\n";
#$url = url 'https://advertise.latimes.com/portal/page/portal/LA%20Tim
+es';
$url = url 'https://advertise.latimes.com/portal/event/portal/LA%20Tim
+es?_pageid=273,1&_dad=portal&_schema=PORTAL&_portlet_ref=273_150732_2
+73_1_1&_eventName_register=';
$res = $ua->get($url);
print $res->is_success,"\n";
print $res->content,"\n";
[download]

Please note that it can successfully access the following URLs:
1. https://advertise.latimes.com;
2. https://advertise.latimes.com/portal/page/portal/LA%20Times;
But it can't handle this URL (which is doing urlrewrite in Portal):
https://advertise.latimes.com/portal/event/portal/LA%20Times?_pageid=273,1&_dad=portal&_schema=PORTAL&_portlet_ref=273_150732_273_1_1&_eventName_register=

I can access this URL using browser without any problem, so what the problem with LWP? Can someone try it on your side and let me know what's your results? I am wondering if I should use Perl 5.6 instead of 5.8?

Edit: g0n - replaced pre tags with code tags

Comment on LWP and Oracle Portal HTTPS site problem Download Code

Replies are listed 'Best First'.
Re: LWP and Oracle Portal HTTPS site problem by chargrill (Parson) on Apr 12, 2006 at 23:22 UTC
When I access your URL of interest, I'm redirected to a page that looks like it's asking me to register, but for some reason doesn't render for me in FireFox. It looks to me like your last `$url` is something you'd access after you've already logged in, and received a cookie from the server. I can't remember if `LWP::UserAgent` handles cookies and logging into websites well, you may want to look into using WWW::Mechanize to have your code go through the process of logging into the website before attempting to access your desired page. --chargrill `$,=42;for(34,0,-3,9,-11,11,-17,7,-5){$.=pack'c'=>$,+=$_}for(reverse s +plit//=>$ ){$%++?$ %%2?push@C,$_,$":push@c,$_,$":(push@C,$_,$")&&push@c,$"}$C[$# +C]=$/;($#C >$#c)?($ c=\@C)&&($ C=\@c):($ c=\@c)&&($C=\@C);$%=$\|;for(@$c){print$_^ +$$C[$%++]}` [download]	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: LWP and Oracle Portal HTTPS site problem
by chargrill (Parson) on Apr 12, 2006 at 23:22 UTC

When I access your URL of interest, I'm redirected to a page that looks like it's asking me to register, but for some reason doesn't render for me in FireFox. It looks to me like your last $url is something you'd access after you've already logged in, and received a cookie from the server.

I can't remember if LWP::UserAgent handles cookies and logging into websites well, you may want to look into using WWW::Mechanize to have your code go through the process of logging into the website before attempting to access your desired page.

--chargrill

$,=42;for(34,0,-3,9,-11,11,-17,7,-5){$*.=pack'c'=>$,+=$_}for(reverse s
+plit//=>$*
){$%++?$ %%2?push@C,$_,$":push@c,$_,$":(push@C,$_,$")&&push@c,$"}$C[$#
+C]=$/;($#C
>$#c)?($ c=\@C)&&($ C=\@c):($ c=\@c)&&($C=\@C);$%=$|;for(@$c){print$_^
+$$C[$%++]}
[download]

[reply]
[d/l]
[select]