in reply to Check website has update file using www::mechanize

See sub LWP::UserAgent::resume
  • Comment on Re: Check website has update file using www::mechanize

Replies are listed 'Best First'.
Re^2: Check website has update file using www::mechanize
by perlmad (Sexton) on May 25, 2016 at 06:53 UTC

    Is this Possible with Http::Response ???

    If yes, then how can get the last modified information from http response ???

[reply]

      HTTP has provisions for not sending data if it is younger than a given timestamp. See the ->mirror method of LWP::UserAgent and/or the If-Modified-Since header of HTTP.

[reply]
[d/l]
[select]

        Which is by no means a guarantee that the data did or did not change. I deal with government data all the time, and their sites just list the ZIP/Excel/CSV/PDF files. You actually have to fetch the files in order to check if they changed (or their content changed).

        My approach is

        • Read persistent file with ZIP/CSV file checksums
        • Read site and parse links
        • For each link with a file I want/know
          • Fetch file into memory
          • Calculate SHA256
          • Compare to previous SHA256
          • same and next
          • save file
          • store SHA256
          • log/mail/other action(s)

        Enjoy, Have FUN! H.Merijn
[reply]

        Yeah it's working but I still have a same problem 

        my $res=$mech>mirror('download_link');
print " response is :",$res,"\n\n"; # no content

        [download]

        I got download file when i ran but I need whether the file is updated or not, if updated then download otherwise just drop a message

[reply]
[d/l]

In Section Seekers of Perl Wisdom