in reply to How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?

Browsers have no reliable event saying "JS has finished executing", because there might be a timer constantly executing functions or callbacks triggered by returning AJAX requests ... and so on.

Note also that JS can run in parallel.

So you have to tell Mechanize to wait for an DOM-element to appear which represent the state you consider "finished".

When working with WWW::Mechanize::Firefox many years ago I used this Wait-until-an-element-appears approach.

You have to see how this best translates to PhantomJS and your use case.

Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery

  • Comment on Re: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?

Replies are listed 'Best First'.
Re^2: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?
by Darth Ed (Beadle) on Aug 21, 2021 at 07:13 UTC
    So you have to tell Mechanize to wait for an DOM-element to appear which represent the state you consider "finished".

    Sure, I get that, but I haven't been able to figure out how to do that with WWW::Mechanize::PhantomJS. I don't see an is_visible method listed in the POD.... I'd be fine with just setting a timer for 3 seconds, but, again, I don't see how to do that either.

[reply]
[d/l]
[reply]
[d/l]
        Note that WWW::Mechanize::PhantomJS is not under development anymore. The program phantomjs also is not developped anymore.
        It's not? I was not aware of that. I don't see anything in the WWW::Mechanize::PhantomJS POD about that, and the most recent release was less than a year ago....

        You're certainly correct about the phantomjs program. I now see that notice at the top of https://phantomjs.org/.

        Unfortunately, I need this to work on a fairly old version of Linux (CentOS 6), so I don't think WWW::Mechanize::Chrome is an option. Maybe WWW::Mechanize::Firefox will work though....

[reply]
      Using one of the selectors in WWW::Mechanize::PhantomJS to find a "finished" characteristic (DOM element or attribute or combination or ...) should return an empty result unless it exists. (See all EXTRACTION-METHODS)

      And even if this didn't work, you could ->eval your own JS-code which does those checks in a loop for you and only returns after success.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

[reply]
[d/l]

      Maybe just use sleep to wait for three seconds?

[reply]
        Maybe just use sleep to wait for three seconds?
        Sleep between the call to the get method and the call to the decoded_content method? I tried that. No change to the content.

        Is there a different arrangement of methods you recommend? Do you have an example that does something like this?

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom