sugu has asked for the wisdom of the Perl Monks concerning the following question:
I'm using Lwp::useragent and Mechanize for getting source page of websites but for this website(https://camelcamelcamel.com/) when runs the code it shows 500 error. I can't able to figure out where is my mistake. I don't know whether my mistake in cookie or useragent, can someone help me with this using Lwp::useragent itself...Thank you in Advance.
use strict; use LWP::UserAgent; use HTTP::Cookies; my $url = "https://camelcamelcamel.com/"; my $ua=LWP::UserAgent->new(); $ua->agent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/2010010 +1 Firefox/46.0"); my $cookie = HTTP::Cookies->new(file=>$0."_cookie.txt",autosave=>1); $ua->cookie_jar($cookie); my $req = HTTP::Request->new(GET=>"$url"); $req->header("Content-Type"=> "application/x-www-form-urlencoded"); $req->header("Accept"=> "text/html,application/xhtml+xml,application/x +ml;q=0.8,*/*;q=0.7"); my $res = $ua->request($req); $cookie->extract_cookies($res); $cookie->save; $cookie->add_cookie_header($req); $res->header("Content-Type"=> "application/xml; charset=utf-8"); my $code=$res->code; print "Code::$code\n";
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re: Can't able to get source page 500 error.
by 1nickt (Canon) on Jun 08, 2017 at 12:53 UTC | |
by sugu (Initiate) on Jun 08, 2017 at 13:38 UTC | |
by 1nickt (Canon) on Jun 08, 2017 at 13:47 UTC | |
by Corion (Patriarch) on Jun 08, 2017 at 13:56 UTC | |
|
Re: Can't able to get source page 500 error.
by mrguy123 (Hermit) on Jun 08, 2017 at 12:44 UTC | |
|
Re: Can't able to get source page 500 error.
by amitsq (Beadle) on Jun 08, 2017 at 13:39 UTC |