Getting Error 503 with LWP::UserAgent

mingke has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I used LWP::UserAgent to retrieve HTML content of one page and it worked. All of a sudden same code does not work and I get error 503 instead. What might be wrong?

This is the page:
http://www.pubmedcentral.nih.gov/tocrender.fcgi?action=cited&tool=pubmed&pubmedid=17061271

And this is my code:

    

$pmc_string = "http://www.pubmedcentral.nih.gov/tocrender.fcgi?action=
+cited&tool=pubmed&pubmedid=17061271"; 
 
my $ua = LWP::UserAgent->new; 
$ua->timeout(10); 
$ua->env_proxy; 
 
my $response = $ua->get($pmc_string, 
   'User-Agent' => 'Mozilla/5.0 [en] (Windows; U; Windows NT 6.0)', 
   'Accept' => 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, i
+mage/png, */*', 
   'Accept-Charset' => 'iso-8859-1,*,utf-8', 
   'Accept-Language' => 'en-US', 
); 
if ($response->is_success) { 
print $response->content; 
} 
else {die $response->status_line;}
[download]

BTW, LWP::Simple does not work either. Same error 503.

Also, this external website is able to grab their HTML OK; http://web-sniffer.net/?url=http%3A%2F%2Fwww.pubmedcentral.nih.gov%2Ftocrender.fcgi%3Faction%3Dcited%26tool%3Dpubmed%26pubmedid%3D17061271&submit=Submit&http=1.1&rawhtml=yes&type=GET&uak=0

What is wrong with my code? Thanks!

Comment on Getting Error 503 with LWP::UserAgent Download Code

Replies are listed 'Best First'.
Re: Getting Error 503 with LWP::UserAgent by pc88mxer (Vicar) on Jun 16, 2008 at 05:00 UTC
A 503 response means that service is temporarily unavailable and you should try again later. For what it's worth, your code works for me. Have you been hitting that server with a lot of requests? One reason you could be getting this error but other methods of getting the page succeed is that the server is intentionally throttling your ip address because you have been making too many calls to it.	[reply]
Re: Getting Error 503 with LWP::UserAgent by kabeldag (Hermit) on Jun 16, 2008 at 06:59 UTC
Before I finalize my scripts that include LWP, I always use the following: `use LWP::Debug qw(+);` [download] It is good for displaying the HTTP request process. If you have to use `$ENV{'HTTP_PROXY'}`, then I guess you have to, it's not that big a deal. My personal preference is to use the LWP::UserAgent proxy() method. I just don't trust environment variables :P	[reply] [d/l] [select]
Re: Getting Error 503 with LWP::UserAgent by ikegami (Patriarch) on Jun 16, 2008 at 05:04 UTC
Works for me. 503 sounds like an error generated by LWP. What's the whole status line? I suspect you're having problems getting through your proxy, but I don't know anything about fixing that.	[reply]