in reply to Re: Fast efficient webpage section monitoring
in thread Fast efficient webpage section monitoring

Are you referring to the if-modified-since option in curl? Unless I'm mistaken, the site doesn't seem to be issuing this part of the header (the "Last-Modified" line is missing from curl -I http://...)
  • Comment on Re^2: Fast efficient webpage section monitoring

Replies are listed 'Best First'.
Re^3: Fast efficient webpage section monitoring
by BrowserUk (Patriarch) on Apr 02, 2016 at 18:59 UTC
    the "Last-Modified" line is missing from curl

    That's one I was thinking about; there are other that might be useful if the server provides them:

    1. Age:The age the object has been in a proxy cache in seconds. Eg. Age: 12
    2. Content-MD5:A Base64-encoded binary MD5 sum of the content of the response. Eg. Content-MD5: Q2hlY2sgSW50ZWdyaXR5IQ==
    3. Content-Length The length of the response body in octets (8-bit bytes). Eg. Content-Length: 348
    4. ETag:An identifier for a specific version of a resource, often a message digest. Eg. ETag: "737060cd8c284d8af7ad3082f209582d"

    Basically, look at what headers are returned and check them from consecutive requests and look for anything that doesn't change per-request; but does change when the content changes.

    Just a thought.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom