LWP Script suddenly stopped working...

fatmcgav has asked for the wisdom of the Perl Monks concerning the following question:

Hi there,

I'm having some issues with some strange behaviour on one of my perl monitoring scripts.

The script uses LWP::UserAgent to request a given url and check the results...

This afternoon, the script has started failing with '404 not found'...

I've checked everything I can think of on the server with no success. The script is still working perfectly on another server...

The HTTP response content looks like:

LWP Returned... Checking response
$VAR1 = bless( {
                 '_protocol' => 'HTTP/1.0',
                 '_content' => '<?xml version="1.0" encoding="iso-8859
+-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>
',
                 '_rc' => 404,
                 '_headers' => bless( {
                                        'connection' => 'close',
                                        'client-response-num' => 1,
                                        'date' => 'Tue, 22 May 2012 12
+:20:13 GMT',
                                        'client-ssl-cert-issuer' => '/
+C=US/O=Equifax/OU=Equifax Secure Certificate Authority',
                                        'client-ssl-cipher' => 'AES256
+-SHA',
                                        'client-peer' => '67.215.65.13
+2:443',
                                        'content-length' => '345',
                                        'client-date' => 'Tue, 22 May 
+2012 12:20:12 GMT',
                                        'content-type' => 'text/html',
                                        'client-ssl-cert-subject' => '
+/serialNumber=u1qcmJQFwrxSq31GeQxJEn9-PfyfU0Fp/C=US/O=*.opendns.com/O
+U=GT55236522/OU=See www.rapidssl.com/resources/cps (c)10/OU=Domain Co
+ntrol Validated - RapidSSL(R)/CN=*.opendns.com',
                                        'title' => '404 - Not Found',
                                        'server' => 'OpenDNS Guide'
                                      }, 'HTTP::Headers' ),
                 '_msg' => 'Not Found',
                 '_request' => bless( {
                                        '_content' => '',
                                        '_uri' => bless( [
                                                           bless( do{\
+(my $o = 'https://cms.wag.card.co.uk/CMSBackOffice')}, 'URI::https' )
+,
                                                           undef
                                                         ], 'URI::URL'
+ ),
                                        '_headers' => bless( {
                                                               'user-a
+gent' => 'libwww-perl/5.836',
                                                               'accept
+-language' => 'en-gb, en',
                                                               'accept
+-encoding' => 'gzip, x-gzip, deflate, x-bzip2'
                                                             }, 'HTTP:
+:Headers' ),
                                        '_method' => 'GET',
                                        '_uri_canonical' => $VAR1->{'_
+request'}{'_uri'}[0]
                                      }, 'HTTP::Request' )
               }, 'HTTP::Response' );
[download]

Any ideas???

Cheers Gavin

Comment on LWP Script suddenly stopped working... Download Code

Replies are listed 'Best First'.
Re: LWP Script suddenly stopped working... by ww (Archbishop) on May 22, 2012 at 12:57 UTC
Sorry to offer such a generic response, especially if you've already considered the stock possibility for problems described with the words "suddenly stopped working... ." But, nonetheless, the standard path to a solution is: Find out what changed on the server. ...is the script still present and unmodified? ... was something upgraded/changed in any way that might impact? If you took the precaution of maintaining a complete manifest of server files, read the entire host again... and compare (for new or missing files) Ditto re your local host Check your logs? (...see anything out of the ordinary?)	[reply]
Re: LWP Script suddenly stopped working... by aaron_baugher (Curate) on May 22, 2012 at 13:10 UTC
As ww said, the problem is too generic to offer much help. When you say the script is working perfectly on another server, do you mean it's running on another server and checking this same host successfully, or that it's checking another host successfully? One thing I noticed: should a header called `server` be set to something like `OpenDNS Guide`? That seems odd. For what it's worth, sometimes servers will be (mis)configured to return error codes if they don't receive certain headers or header values. I've seen this more often with Windows servers, for whatever reason. For instance, someone might be trying to serve different content to different browsers, and they didn't set up a default version, so when it sees "libwww-perl/5.836", it just fails. Or it might refuse to send pages to a 'browser' that doesn't send an Accept-Language header. You see odd stuff like that now and then when you're trying to automate stuff like this. If you don't have control of the server you're trying to monitor, your options are limited. Basically, you can try to emulate a browser as much as possible, and try sending different combinations of headers, until something works. Aaron B. Available for small or large Perl jobs; see my home node.	[reply] [d/l] [select]