Retriving search results with WWW::Mechanize

palkia has asked for the wisdom of the Perl Monks concerning the following question:

hello

I tried to use WWW::Mechanize to retrieve the content of a webpage,
which contains a small search engine, a search button, and the results.

The thing is I can't seem to be able to read the "results" of the search.
(note that the page got a search default, so it shows some search results even when the page is just loaded (before you can choose your search))

I'm guessing this is because the "get" function doesn't "wait" for the results to be loaded and only retrieves the initial content,
or that maybe the page uses some sub-page or I-frame (I know very little about these things), which (as far the "get" concerns) isn't part of the page's content.

Because I can't get the search results, I can't even be sure if I'm successful in launching the "search" command of the search engine after in finishes the default search.

The page in question is this one
but beware as it is not in english (so don't think there's an error if you can't see the characters correctly ^^)

What can I do to retrieve the search results ? (wasted hours trying to figure out the damn thing)
10q very much

Comment on Retriving search results with WWW::Mechanize

Replies are listed 'Best First'.
Re: Retriving search results with WWW::Mechanize by moritz (Cardinal) on Apr 17, 2011 at 09:01 UTC
Here's a simple rule: Go to the result page, and view the page source (Ctrl+U in the standard firefox). If you see the information you want in there, it's most often just a matter of tweaking the data extraction from WWW::Mechanize. If not, you're in trouble (most often: javascript). Perl 6 - second systems done right	[reply]
Re: Retriving search results with WWW::Mechanize by Corion (Patriarch) on Apr 17, 2011 at 07:38 UTC
... that maybe the page uses some sub-page or I-frame (I know very little about these things), which (as far the "get" concerns) isn't part of the page's content. Maybe now is the time to learn about such things? It may come as a surprise to you, but that page is using Javascript. Please do read what the WWW::Mechanize documentation has to say about Javascript support.	[reply]
Re: Retriving search results with WWW::Mechanize by Anonymous Monk on Apr 17, 2011 at 00:10 UTC
What can I do to retrieve the search results ? (wasted hours trying to figure out the damn thing) See How can I use perl as an internet browser ?	[reply]