Dear all

I originally submitted this question here but, after some research, have come to realise that I formulated the question wrong, so am trying again

I'm simply automating a search on a citations web-page using LWP::UserAgent.

When I submit the search online, I get a full page back immediately, with some results on it, but the page is set to refresh automatically, and if I wait a couple of seconds, the page updates exactly how many results I have available (it only shows ten per page).

When I submit using UserAgent and HTTP::Request::Form I get a results page in full, but it's the initial page, which is the page waiting to refresh with ALL the results. It's weird because it's only 10 per page, and the refresh is just to tell you the full number of records found. I cannot figure out what to do about the refresh, because if I resubmit the search I get back the very same initial results page, but I don't know how to 'refresh' for the new page from UserAgent...

What I really dont understand is that I want to download ALL the records via a button, but if I only get the initial web-page, then the number of records is small, i would have thought that especially because the server itself should be continuing to find records, even if I 'press' the button, it would thereby download ALL the records the server has found, and not necessarily the ones found when the results page was first submitted.

Does this make sense?

Thanks
Sam

janitored by ybiC: Changed <a href...> nodelink to PM-style link a'la [id://392513|here] as per Monastery specification in PerlMonks FAQ

In reply to refreshing a page/session without re-submitting search by seaver

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.