jdlev has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks!

I'm working on a perl scraper using www::mechanize, but need it to execute a line of jQuery code to pop information about an element on a page. I could either use jquery or not, it doesn't really matter, I just figured jquery might be a bit more concise. Basically, the code I want perl to click is this, so how would I go about doing that? 

<div class="slick-cell l1 r1">
  <a href="#" onclick="return Game.ContestPop(33445566, true);" class=
+"fr"> Contest 33445566 </a>
</div>

[download]

Thanks in advance monks :D

I love it when a program comes together - jdhannibal

Replies are listed 'Best First'.
Re: jQuery & Perl?
by Corion (Patriarch) on Sep 22, 2014 at 17:14 UTC
[reply]
      Well...farfignuggen..from the link you gave me (thanks btw)

      "How can WWW::Mechanize use javascript? You can't. JavaScript is entirely client-based, and WWW::Mechanize is a client that doesn't understand JavaScript. See the top part of this FAQ."

      So can you suggest a way to identify the link and have mechanize click it? I think I 'might' be able to get it done with mech->link somehow? For instance, between the a tags, it has something like "ContestPop(334455)" as part of the link. Is there a way that if perl parses the content from the page, and finds anything like "ContestPop(334455)" in a link, it automatically follows it? Keep in mind the contest pop number could change, so what type of operator could I use with mech->link to have it follow a link that 'sort of' matches what I'm looking for?

      I love it when a program comes together - jdhannibal
[reply]

        It's not that easy. The Javascript function ContestPop() could do many things that are not readily understood by Perl, unless Perl understood Javascript, which it doesn't.

        I suggest you learn about HTTP and find out what request is made when you (as a human) click on the Javascript link. For example the Mozilla Live Headers Extension can show you what HTTP requests are made. Most browsers have such tools built into them so you can conveniently analyze what goes over the wire.

        After you've found out what data gets send, all that remains is replicating that using Perl.

        As an alternative, there are implementations of the WWW::Mechanize API that "understand" Javascript by running the Javascript in a browser. Two examples are WWW::Mechanize::Firefox and WWW::Mechanize::PhantomJS. But these both need the respective browser installed. Also, they still require you to understand the difference between Javascript and Perl, and you will also need to understand where HTTP comes into play.

[reply]
[d/l]

Back to Seekers of Perl Wisdom