Hi All

The task I have at hand is as follows:
There is a webpage which changes periodically, and when it does, it displays a button which I need to click as fast as possible (basically before anybody else does).
The way I have implemented this so far is to write a script which uses curl to download the page, grep it for the "button" part, and if it finds nothing download again until change is found.
When change is found I also use curl to press the button which works ok as far as I'm concerned (although no doubt it could be improved).
The part I am not too pleased with is the one that monitors the webpage:
-Firstly it uses a lot of bandwidth, comparatively, since it's endlessly downloading the same thing over and over again...
-Secondly it is relatively resource greedy.

What I would like help with is making a decision as to whether by using LWP, HTTP::Monitor, HTTP::Tiny, or any similar module I might be able to improve on this. Or perhaps simply switching to wget. I'm quite ignorant indeed as to the performances of all these tools for this kind of job.

Another thing I was wondering is if there is a way to save time when the webpage actually does change, by interrupting the download for instance. I know LWP provides some kind of "loop back" as the download is proceeding, but I don't know quite how I could implement that.

I'm sorry if this question is out of scope, or too wide, or not well asked. Any help will be appreciated.

Thank you!
Mark.

In reply to Fast efficient webpage section monitoring by Marcool

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.