rustycar has asked for the wisdom of the Perl Monks concerning the following question:

I'm working up a script for testing our product that has a web-based setup interface.

I've gotten pretty far, but now I've hit a point that has me stumped. I'm using the conncach thing (even though, in hindsight, I don't think I need it) along with WWW::Mechanize - which, if I RTFM'd correctly, does Keep-Alive without even needing ConnCache (but, I could be more confused than usual here).

What I do is post a form to the server, and it comes back with a page that says, basically, "working...". The server sends the complete page, waits a bit (up to 10 seconds), then sends another 'working...' page, (rinse, repeat until the server decides its done), and then FINALLY sends a 'real' page.(see update below - this is wrong)

On a different project, I found that I had no problem getting the 'real' page that followed the 'snooze' page(s) simply by ignoring the fact that they were there. (update - must be a different mechanism, sorry) However, here I'm not getting any further pages - Mechanize seems to think its done after the first 'working..'.

(The server is NOT sending javascript to the client to make the 'snooze' page look active. Its simply sending pages with 'working.' or 'working..' or 'working...' to make it look like the dots are moving.)(again, almost right, see update)

If more detail on the actual html is needed, I can get the page source (or wireshark captures, whatever) tomorrow when I'm back at work....

In case it matters, I'm using the Mechanize::click_button method to post the form.

So, the question: is there any trick I need to pull, or do I need to explicitly get Mechanize to reload something, to get Mechanize to wait for the 'real' page?

update

Thanks to all for the replies. And sorry I didn't include code or html last time - I was at home and the stuff is at work (where I am now) - and I was hoping there was a generic answer.

Ok - on to the point!

Mr/Ms Anonyymous Monk pointed to an article to read. The point of the article is to use a refresh tag to tell the browser to try again in a bit. Which turns out is exactly what the page does, so my calling it server push was wrong! Ouch. Anyway, here's the relevant part of the 'working' page, which shows that indeed we're doing 'client pull':
Cache-Control: no-cache Connection: Keep-Alive Pragma: no-cache <blahblahblah...headers..headers> Refresh: 2; URL=ProgressPage?sid=3798173575391795693&indicator=2 Title: WebView 5.4 <html> <head> ...blahblahblah...html... <META http-equiv="Refresh" content="2; URL=ProgressPage?sid=3798173575 +391795693&indicator=2"> <title>WebView 5.4</title> ...blah... </head> <body onLoad="" onKeyPress=""> <table height="100%" align="center"> <tr><td valign="center" align="center"> Please wait, loading .&nbsp;&nbsp;</td></tr> </table></body></html>

So, I think I'll do this:
# click the button, then get a 'working.' page: $ans = $mech->click_button( input => $inputobj ); while ($mech->response() =~ m/working/) { sleep(5); $ans = $mech->reload(); }
Thanks to all! Mark this one solved, unless someone wants to tell me why the above is a bad solution :-)

Final (?) update: I've updated the text in the original query, striking out a few errors. Just wish this page had a 'preview' button like the original posting page did... (I'm assuming its bad form to delete original text when you later realize you blew it :-)

Replies are listed 'Best First'.
Re: Having trouble with keep-alive, or server push pages, or something
by Lawliet (Curate) on Jul 23, 2008 at 00:41 UTC

    You could use $mech->reload();

    Perhaps sleep for 5 seconds, reload, check if its the page, continue or repeat if necessary

    <(^.^-<) <(-^.^<) <(-^.^-)> (>^.^-)> (>-^.^)>
Re: Having trouble with keep-alive, or server push pages, or something
by ikegami (Patriarch) on Jul 23, 2008 at 04:50 UTC

    Keep-Alive has nothing to do with server push. It's a non-standard method of allowing HTTP/1.0 clients to make multiple requests over one connection. (It's the default in HTTP/1.1 and clients which don't support it must send "Connection: close".)

    As for why Mechanize doesn't work with your server's output, it would help if we had some code to work with, particularly the server's output.

Re: Having trouble with keep-alive, or server push pages, or something
by Anonymous Monk on Jul 23, 2008 at 04:21 UTC
    What makes that page that says "working", "working.", "working.." ... do that?
    Its not server push or keep-alive, its "something".
    Examine the html.
    grep WWW-Mechanize
    Read this article.