nysus has asked for the wisdom of the Perl Monks concerning the following question:

I'm kicking the tires of WWW::Mechanize::Chrome. I'm finding I need to insert stuff like sleep 3; after each page request to wait for a page load before filling out a form. Without sleep, the page doesn't have time to load and I can't input data into the fields and submit the form. I don't recall having this problem with WWW::Mechanize::Firefox as it seemed to have some kind of sync feature. Is there a similar way to automatically wait for the page to load using WWW::Mechanize::Chrome?

$PM = "Perl Monk's";
$MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
$nysus = $PM . ' ' . $MCF;
Click here if you love Perl Monks

Replies are listed 'Best First'.
Re: Waiting for page to load with WWW::Mechanize::Chrome
by LanX (Saint) on May 29, 2018 at 00:00 UTC
    Nowadays "Loaded" is a matter of definition, because various JS-events can cause further actions.

    Back then when working with WMF I had to check if the DOM element I needed was already available, before I continued.

    Cheers Rolf
    (addicted to the Perl Programming Language and ☆☆☆☆ :)
    Wikisyntax for the Monastery

Re: Waiting for page to load with WWW::Mechanize::Chrome
by Corion (Patriarch) on May 29, 2018 at 06:58 UTC

    The kinds of events that Chrome makes available to the outside world are different from the events that Firefox makes available.

    If you need to wait 3 seconds or more, then most likely as LanX already said, some Javscript only runs after the page is considered complete (by the browser) and then loads additional information. LanX's approach of waiting for a page element is the approach I think is most fruitful. Unfortunately, WWW::Mechanize::Chrome doesn't have the ->wait_until_* methods of WWW::Mechanize::Firefox yet, but maybe you can submit a patch for that?

      This isn't a patch but one possible solution. I wrote a wrapper for WWW::Mechanize::Chrome in Moose. Note that that get method is overridden. Every 1/10th of a second it checks to see if new elements have been loaded on the page. If there were, it waits another 1/10th of a second. The length of time can be increased for slower connections.

      package WWW::Mechanize::Chrome::MooseWrapper ; use Moose; use MooseX::NonMoose; use Time::HiRes qw(usleep); use File::Temp 'tempdir'; use Log::Log4perl qw(:easy); use Data::Dumper qw(Dumper); use namespace::autoclean; extends 'WWW::Mechanize::Chrome'; sub FOREIGNBUILDARGS { my $class = shift; my %args = ( @_ == 1 ? %{ $_[0] } : @_); # set default launch args $args{launch_exe} = $args{launch_exe} || "/Applications/Go +ogle Chrome.app/Contents/MacOS/Google Chrome"; $args{launch_arg} = $args{launch_arg} || [ '--disable-noti +fications' ]; $args{data_directory} = $args{data_directory} || tempdir(); $args{incognito} = exists $args{incognito} ? $args{incognito} +: 1; return %args; } ### Public methods ### sub scroll_to_bottom { my $s = shift; $s->eval( 'window.scroll(0,document.body.scrollHeight)' ); } sub scroll_to_top { my $s = shift; $s->eval( 'window.scroll(0,0)' ); } sub get_element_count { my $s = shift; my ($el_count) = $s->eval( 'document.getElementsByTagName("*").lengt +h' ); return $el_count; } around get => sub { my $orig = shift; my $s = shift; my $url = shift; $s->SUPER::get($url); my $el_count = $s->get_element_count; # wait 1/10th sec for more of the page to load usleep 100000; while ($el_count != $s->get_element_count) { # wait 1/10th sec for more of the page to load usleep 100000; $el_count = $s->get_element_count; } }; ### Private methods ### __PACKAGE__->meta->make_immutable; 1; # Magic true value

      $PM = "Perl Monk's";
      $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
      $nysus = $PM . ' ' . $MCF;
      Click here if you love Perl Monks