chuck_norris has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, I'm trying to get search query results from a web database. I'm using the WWW::Mechanize module with max_redirect = 100, but still get stuck in the middle of the process. I'm getting an interim HTML page which says that the search is in progress. How can I skip that page and get the final results page? here is the code:
my $agent = WWW::Mechanize->new(max_redirect=>100); ### First call - authentication $agent->get('http://search.library.nhs.uk/nhs_sse/zengine?'); ### Second call - run the search query $agent->get('http://search.library.nhs.uk/nhs_sse/zengine?VDXactio +n=ZSearchAdvanced'); $agent->form_number(2); $agent->set_fields('search_term_row1'=>'science', 'attr_type1_row1'=>'4'); $agent->click(); print $agent->content;
Thanks in advance!

Replies are listed 'Best First'.
Re: HTML problem
by svenXY (Deacon) on Jan 02, 2008 at 11:10 UTC
    Hi,
    consider using WWW::Mechanize::Shell to interactively use WWW::Mechanize like a text browser, then use the script method to dump what you did interactively as perl code - that should show you how to get around this problem.
    Regards,
    svenXY