Re: Perl https parser

As derby suggested above, it looks like a proxy configuration issue. As for the "microsoft xp browser", I assume you're asking how to set the agent to look like its coming from Internet Explorer? If so, you need to determine what user agent string you want to send (it varies between windows installations), and then simply supply it to the agent method. You can basically provide it any string:

$ua->agent('My own custom user agent');
$ua->agent('Anything you want!');
[download]

__________
Systems development is like banging your head against a wall...
It's usually very painful, but if you're persistent, you'll get through it.

Comment on Re: Perl https parser Select or Download Code

Replies are listed 'Best First'.
Re^2: Perl https parser by onlyakhila (Initiate) on Jul 16, 2007 at 20:03 UTC
Thank you for your replies. I am trying to get the content from a web page (which has XML feeds in RSS 1.0, 2.0 and atom) and pass it through a RSS Parser to extract data which is why i am using XML::RSS. Firstly i was not able to retrive the RSS feed from the https webpage but i am able to do so for http sites. If i dont set a proxy i am not able to get to http sites either. Secondly I tried to pass the rss parser to a http site (http://ww.cnn.com/) and got an error: no element found at line 1, column 0, byte -1 at I:/Perl5.8.8.817/lib/XML/Parser.pm line 187 The code for XML::RSS is: #!/usr/local/bin/perl $ENV{"HTTP_PROXY"} = "http://http-proxy:xx"; $ENV{"HTTP_PROXY_USER"} = "xxxx"; $ENV{"HTTP_PROXY_PASS"} = "xxxx"; use LWP::UserAgent; use XML::RSS; $ua = LWP::UserAgent->new; $req = HTTP::Request->new(GET => 'http://www.cnn.com/'); $ua->env_proxy(); $ua->agent('Mozilla/5.0'); $res = $ua->request($req); if ($res->is_success) { #print ($res->content); printf "fetched %d bytes\n", length($res->content); } else { print "Error: " . $res->code . " " . $res->message; } my $rss = new XML::RSS; $rss->parse($content); #print "rss is $rss\n"; [download] I have read that XML::RSS supports all forms of RSS feed, is that so? I would appreciate any help i can get on building an RSS parser in perl for https sites. Thank you, Akhila	[reply] [d/l]

Replies are listed 'Best First'.

Re^2: Perl https parser
by onlyakhila (Initiate) on Jul 16, 2007 at 20:03 UTC

#!/usr/local/bin/perl
$ENV{"HTTP_PROXY"} = "http://http-proxy:xx";
$ENV{"HTTP_PROXY_USER"} = "xxxx";
$ENV{"HTTP_PROXY_PASS"} = "xxxx";
use LWP::UserAgent;
use XML::RSS;

$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => 'http://www.cnn.com/');
$ua->env_proxy();
$ua->agent('Mozilla/5.0');
$res = $ua->request($req);
if ($res->is_success) {
#print ($res->content);

printf "fetched %d bytes\n", length($res->content);
} else {
  print "Error: " . $res->code . " " . $res->message;
}

my $rss = new XML::RSS;
$rss->parse($content);
#print "rss is $rss\n";
[download]

[reply]
[d/l]