in reply to Re: WWW::Mechanize::Firefox latency using RemoteObject
in thread WWW::Mechanize::Firefox latency using RemoteObject

OK, that ->selector() syntax looks interesting.

Do you see what I'm saying about how long each of the transactions to the browser and back takes, though? I'm curious if you have any thoughts on what's causing that bit. As I said, it seems a wee bit odd that the system is idle instead of cpu or I/O bound.

  • Comment on Re^2: WWW::Mechanize::Firefox latency using RemoteObject

Replies are listed 'Best First'.
Re^3: WWW::Mechanize::Firefox latency using RemoteObject
by Corion (Patriarch) on Apr 04, 2011 at 07:06 UTC

    This is just as you suspected - each attribute access involves at least one roundtrip over TCP from Perl to Firefox and back. There is little you can do except avoid accessing Javascript data from Perl, or to make bulk requests.

    Likely, the CPU time gets split up between Firefox and Perl and the kernel for the Network, and I'm not sure how your monitoring accounts for time spent in the kernel.

    Update: If you start to optimize your application and try to reduce the accesses to Javascript objects, the object bridge has some counters that might be of help:

    stats => { roundtrip => 0, # total number of roundtrips fetch => 0, # number of attribute fetches store => 0, # number of attribute stores callback => 0, # number of callbacks triggered },
    use Data::Dumper; my $repl = $mech->repl; warn Dumper $repl->{stats};
      Thanks for the stats tip. That will help a bunch.

      So, I did just put my code in a loop. It sits basically pounding on the roundtrip and fetch operations in the stats. Even when this is happening, the CPU is never more than about 25% utilized, total, including perl, firefox and the kernel.

      I'm measuring using vmstat, which accounts for system time nicely. I also did some kernel profiling to make sure nothing in there looked funny. Nothing really sticks out there.