help with LWP

fullmotiondesigns has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, I'm stumped by a problem using LWP:UserAgent and was wondering if someone can provide some help. I'm on OS X 10.4 and when running the following script (as a cron job every 10-15min) after a few hours I get an http 500 Invalid Argument and it doesn't seem to go away until after I shut down the machine and restart. I get a http 200 is_success response up until this point. I don't have access to the server logs to check what the server is responding to, but if I curl it from the command line when I am receiving the error I get the web page I expect (http 200). Any ideas on what I can do to troubleshoot this problem?

#!/usr/bin/perl

require LWP::UserAgent;
$WebURL = "http://chili.testxyz.com:10172/pepper/login.jsp";
$ua = LWP::UserAgent->new;
$ua->timeout(30);
$ua->agent("Mozilla/5.0");
$ua->protocols_allowed(undef);

$response = $ua->get($WebURL);
$content = $response->content;

print $content;
[download]

Comment on help with LWP Download Code

Replies are listed 'Best First'.
Re: help with LWP by pc88mxer (Vicar) on Mar 03, 2008 at 23:12 UTC
Some questions... Are there any proxies between you and the url your are trying to access? Are you sure that your cron jobs are not overlapping each other (i.e. one is not starting up while the other one is running)? By shutting down and restarting, do you mean your OS X (client) machine?	[reply]
Re^2: help with LWP by fullmotiondesigns (Initiate) on Mar 03, 2008 at 23:54 UTC
Sorry I should have included those answers as well. There are no proxies between the client machine and the server, there are no other cron jobs running and therefore there shouldn't be any conflict and yes, shutting down and restarting means the OS X client machine.	[reply]
Re: help with LWP by ysth (Canon) on Mar 04, 2008 at 00:12 UTC
I'm wondering if your ipaddr is getting flagged for TOS violation and restarting gets you a new dynamic ip address? Once you start getting the 500, can you access the page via a real browser? -- Online Fortune Cookie Search	[reply]
Re^2: help with LWP by fullmotiondesigns (Initiate) on Mar 04, 2008 at 01:07 UTC
Yes, I'm able to get to the page via Safari or Firefox and I can curl the page from the command line when I'm getting the http 500 invalid message. But, you may be on to something. I moved the machine to another subnet, got a new ip address and it immediately worked again without the 500 error. Strange that I can load it via a browser and can get the url with other tools, but it's intermittent with LWP.	[reply]
Re^3: help with LWP by ysth (Canon) on Mar 04, 2008 at 02:32 UTC
Strange indeed. Does changing the user agent string to be a real one like `Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b3) Gecko/2008021416 Fir +efox/3.0b3` [download] make the error go away once it's started? -- Online Fortune Cookie Search	[reply] [d/l]
Re^4: help with LWP by fullmotiondesigns (Initiate) on Mar 04, 2008 at 19:05 UTC
Re^3: help with LWP by Anonymous Monk on Mar 05, 2008 at 20:00 UTC
(I attempted to reply to the previous post but the text doesn't show up for some reason, so I've bumped this up one response level to see if this works for displaying the text) Sorry, but it doesn't. I restarted the machine and added your user agent string and it failed about 2 hrs later. I'm tempted to try this on a windows machine to see if I get the same problems. Anybody else have any other ideas? Is there a way I can print out the actual request going to the server to see if that is what is causing the problem?	[reply]