Lhamo Latso has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monk,

I have written a crontab Perl program to check whether the company website is available for the customers. We would like to know if there is a problem before the users call to report it.

My code is running on Perl 5.6.1 under RedHat Linux AS v2.1. I am using lwp-request to hit the website.

I am finding that the response time varies at certain times of the day and my timeout causes false failures. I have increased the timeout, but I would like to capture the HTTP error code and know whether it is a timeout or something else.

Here is the code. Thanks in advance! 

#!/usr/bin/perl -w
#       /usr/local/bin/webalert.pl [mail address] [web address]
use strict;
use LWP;
local *MAIL;

# get the mail address if there is one.
my $mailaddr = shift @ARGV;

# get the web address if there is one.
my $webaddr = (shift @ARGV ||"www.perlmonks.org");

my $timeout = 15;
if (system "/usr/bin/lwp-request -t $timeout $webaddr >/dev/null 2>&1"
+) {
  sendmail_or_print()
  }

sub sendmail_or_print {

  # get the mail address if there is one.
  my $hostname = "from ". (`hostname 2>/dev/null` ||"who");

  # get the mail address if there is one.
  if ($mailaddr) {
    open MAIL, "|mail -s \"Web Alert $hostname\" $mailaddr";
    }
  else {
    open MAIL, ">&STDOUT";
    }

  print MAIL "\n\n$webaddr NOT available: " . localtime() ."\n\n";
  print MAIL "$timeout second timeout used.\n\n";
  close MAIL;
  }

[download]

Replies are listed 'Best First'.
Re: website availability ping
by Roy Johnson (Monsignor) on Oct 28, 2003 at 19:34 UTC
    Instead of your system call, use LWP::UserAgent. Here's some code I wrote to do the same thing. 
                ## Request a web page
            use LWP::UserAgent;
            my $ua = LWP::UserAgent->new;
            $ua->timeout($timeout);
            my $response = $ua->get($connection);
            if ($response->is_success) {
                $output = $response->content;
            }
            else {
                $output = $response->status_line;
                $exit_code = $response->code;
            }

    [download]
[reply]
[d/l]
Re: website availability ping
by pg (Canon) on Oct 28, 2003 at 19:38 UTC

    Use LWP::UserAgent, and it does provide error messages sufficient enough for you in this case.

    require LWP::UserAgent;
use Data::Dumper;
use strict;
use warnings;

my $res = LWP::UserAgent->new()->get('http://www.somesite.com/');
print "return code = ", $res->code(), "\n";
print "msg = ", $res->message(), "\n";

    [download]
[reply]
[d/l]
Re: website availability ping
by diakonos (Hermit) on Oct 28, 2003 at 19:04 UTC
    There is a module that may give you a little help on the ping command. Instead of doing a system call you could use NET::Ping. Here is an example: 
    #!/usr/bin/perl -w

use strict;
use Net::Ping;

my $host = "XXX.XXX.XXX.XXX";
my $p = Net::Ping->new();
print "$host is alive.\n" if $p->ping($host);
$p->close();

    [download]
    This module allows you to specify timeout values and also change the type (TCP,UDP, or ICMP) of echo requests that you send. (Up to 6 different protocols) Ex. you cannot ICMP www.micro$oft.com but you can establish a tcp request to the echo port and determine if the server is there. TCP seems to be a little better suited than ICMP but causes more overhead and traffic.

    This module also allows you to do a service check (still not fool proof why HTTP or another service is not responding - that would require coding per service level). It has some return value checking to give you some reason why the response was negative.

    I would use this first before trying to harvest an HTTP error on a remote server which may never reveal itself for various reasons.

    Good Luck

[reply]
[d/l]

      ping does not fit in here at all. That a computer responses to ping, does not mean that the web site resides on it is alive. One the other hand, it is perfectly valid for a web site to be alive on a computer does not response to ping.

      They are just not related.

      Better go straight to deal with the issue by using LWP::UserAgent.

[reply]
        I disagree. In his post he is trying to find problem determination by trying to retrieve a website. If by chance the http request is timing out (DOS, high traffic, etc) he will not receive an http error message besides "timed out". This just says that there was no reply to the request sent. If you read the original post I believe he is trying to get beyond the fact that the server is timing out.

        Continue to use LWP::UserAgent, but dig deeper to see what the problem is. Is the server up? (ICMP call) Is the port listening? (TCP call) Has ICMP been shut down? (Another TCP Call) All which can be accomplished through NET::Ping.

        The fact that an HTTP request times out does not mean that it is an HTTP problem.

        Understand that the word "Ping" as the original author used does not only apply to ICMP packets.

        The problem resolution is definately related.

[reply]
Re: website availability ping
by TVSET (Chaplain) on Oct 29, 2003 at 02:57 UTC
    Additionally to all the answers above, you could grep the content of the result for something that you know exists on the page. For example, the Copyright notice, which is at the bottom of most corporate pages. :)

    Leonid Mamtchenkov aka TVSET

[reply]
[d/l]
Re: website availability ping
by jacques (Priest) on Oct 29, 2003 at 02:26 UTC
    Merlyn has a new article in the latest Linux Mag on exactly this subject. He uses Mech, which is perfect for this task. I think it's worth checking out. (Unfortunetly, you can't get the article on his website.) Mech == WWW::Mechanize
[reply]

Back to Seekers of Perl Wisdom