in reply to WWW::Mechanize and javascript

Even if you can run JavaScript within your script via something like JavaScript::SpiderMonkey, it will still be a challenge to understand the browser's DOM completely. JavaScript itself doesn't know what an "onClick" event is, it's just a language. Only in combination with the browser's DOM you get this powerful browser-scripting ability. I heard that leira is working on something like that.

Until then, I'd recommend using a logging proxy like proxy.pl. Just start it on the command line like 

    ./proxy.pl

[download]

and when it says Server listening at port 8017, point your browser to it via Preferences-Advanced-Proxies, where you click on Manual Proxy Configuration and set the HTTP proxy entry to localhost and port 8017.

With this enabled, the logging proxy will tell you exactly what he browser sends over the wire when you're surfing, and you can simply wrap its output into a WWW::Mechanize script.

Replies are listed 'Best First'.
Re: Re: WWW::Mechanize and javascript
by diotalevi (Canon) on Mar 23, 2004 at 22:59 UTC
[reply]
Re: Re: WWW::Mechanize and javascript
by rhxk (Beadle) on Mar 24, 2004 at 00:03 UTC
    I ran the proxy.pl in regular http:// requests and saw the very nice output. I can certainly use it when I want to know what mozilla is doing & redirect it to a perl script...very nice..thanx...

    However, for this task, that webpage I need to access resides on the https protocol & didn't see proxy.pl STDOUT the https requests.

    should I add localhost:8017 in the mozilla proxy or ?????

    thanx..
[reply]
      Unfortunately, the proxy approach (proxy.pl or HTTP::Recorder) works only on plain HTTP, not in combination with SSL.

      If it's just a single request you're trying to examine, could you possibly wing it by saving the page locally and modify the URL it sends the data to, as described here?

      --saintmike

[reply]
        Actually, the latest version of HTTP::Recorder, 0.03_01, works with SSL.
[reply]

      Instead of the proxy hassle, you may also want to look at Live HTTP Headers (since you're already seem to use Mozilla).

      --
      b10m

      All code is usually tested, but rarely trusted.
[reply]
      how do i get the log file
      <update>
      I tried using the method defined.I executed
      ./proxy.pl
      On doing this, it said Server listening at port 8017
      then as per instructions, i directed the browser's proxy settings to loacalhost with port 8017
      What i did not understand is how do i obtain the log file.
      Please help
      </update>
[reply]
        The page I am acquiring through a WWW::Mechanize object, is on loading calling a javascript code for onload. The code is given below. I have read around, but am not able to figure out how to emulate this javascript function.
        If somebody could please point me in the appropriate direction. 
        <script language="javascript">
    function window_onload() 
    {
document.getElementById("divCal").style.display="block";
document.getElementById("divMessage").style.display="none";
    }
</script>

        [download]
        Update
        Ok understood, as i dont have to display the page, only have to scrape it, the above code is not necessary to be emulated by me.
        Thank you for your help.
[reply]
[d/l]

In Section Seekers of Perl Wisdom