in reply to How to get web creation date from webserver?

I have to get web page creation date from the web server.
Why? It's not necessarily available. It might not even make sense. If the server says "last-modified", you might use that, but that may or may not be present, and that may or may not be what you consider a "creation date".

Perhaps you need to step back and ask yourself why you have this problem. That's sometimes a useful problem solving technique when you've asked a question that is impossible to answer.

-- Randal L. Schwartz, Perl hacker
Be sure to read my standard disclaimer if this is a reply.

  • Comment on Re: How to get web creation date from webserver?

Replies are listed 'Best First'.
Re^2: How to get web creation date from webserver?
by gube (Parson) on Aug 23, 2005 at 05:47 UTC

    Hi all,

    Using httrack, by given url i have download the page and working in offline. I have to go and check the url daily the page as been modified or not. If the page as been modified i have to download else i have to exit. so, for this purpose i want to get the date is it possible help me.

[reply]

      Read the HTTP specification. Specifically, section 14.25, 'If-Modified-Since'.

      You return the 'Last-Modified' timestamp from when you cached the file (or the date you got it, but then you have to deal with generating the date format), and if the file hasn't been modified, and the webserver supports this header, it should return a '304' status message, rather than the full content all over again.

[reply]
      So you don't want to know when the page has been created, you want to know if the page has changed since you have last visited/downloaded it. I don't know of a readymade perl way to do this, but there are a lot of of programs, e.g. webmon.


      holli, /regexed monk/
[reply]
        You could fetch the page with LWP, calculate & store an MD5 checksum, then simply compare the current checksum with the last one.

        Code hastily snipped and sanitised :) 

        use Digest::MD5 qw/md5_hex/;
sub web_MD5 {
    # get MD5 sum of an url
    my $url = shift;
    my $ua = LWP::UserAgent->new(env_proxy => 1,
                 keep_alive => 1,
                 timeout => 30);
    my $response = $ua->get($url);
    unless ($response->is_success) {
      # failed to fetch
      print "Error fetching ", $url, " ",    
      $response->status_line;
    }
    warn "Error while getting ", $response->request->uri,
      " -- ", $response->status_line, "\nAborting";
    unless $response->is_success;
    my $doc = $response->content();
    my $md5 = md5_hex($doc);
    undef $ua;
    return $md5;
}

        [download]
[reply]
[d/l]

        Dear holli,

        Is it possible that any tool for linux. And also, i want to check automatically and based on that i want to run perl file to download. If based on webmon means i have to check daily anyother way to get information automatically the changes made in the page information Thanks.

[reply]

In Section Seekers of Perl Wisdom