in reply to Re^3: WWW::Mechanize::PhantomJS can't click button
in thread WWW::Mechanize::PhantomJS can't click button

Thanks again for all that, bliako.

I don't really need to do a screen shot.  That was just an attempt at an easy way to see what had happened.

I tried your code, and its timing seems to work, thank you!  So that's the timing sorted.

However, I still have problems:
- If I try to login with a dummy email/password (as we've used above), I should get the error message "Email or password not found."...etc, but I don't.  The screen shot shows "Please try again later."..etc, with the email address selected (i.e. blue background).
- If I try to login with a valid email/password, I get the same result as the dummy email/password gives.  It should be logging in and taking me to "https://profile.ccli.com/".
- This "Please try again later."..etc, is the same message I get when I manually try to login with valid or invalid email/password when I have JavaScript disabled in Firefox.

Any ideas what's causing the above problems and how I can resolve them?

Re using Mechanize for scraping, I used to use WWW::Mechanize (not WWW::Mechanize::PhantomJS) for logging in to this same site, and I used Firefox's dev tools to grap the POST info to do that, but last year they changed the site and I couldn't see how to get that working anymore (perhaps just my lack of understanding), and they seemed to be requiring JavaScript, so I installed phantomjs and WWW::Mechanize::PhantomJS.  If I manually go to the site with Firefox with JavaScript disabled, there's a message at the top "JavaScript is disabled. Please enable to use the site.".  Can I ignore that message when it comes to my code, or do you think they might be blocking non-JavaScript agents?

(BTW, here's a shorter URL that I've just realised we can use for this: https://profile.ccli.com/Account/SignIn).

  • Comment on Re^4: WWW::Mechanize::PhantomJS can't click button

Replies are listed 'Best First'.
Re^5: WWW::Mechanize::PhantomJS can't click button
by bliako (Abbot) on Apr 30, 2021 at 07:34 UTC

    You will have better results when you set a user-agent string and/or a file for persisting cookies like so:

    my $mech = WWW::Mechanize::PhantomJS->new(
        'cookie_file' => 'cookies.txt'
);
$mech->add_header(
         # use a proper string from somewhere or your own browser
        'User-Agent' => 'Mozilla/5.0 ... bla bla bla'
);

    [download]

    I said "and/or cookie file" because I could not find the cookie file on current dir - i don't know where it may be stored if at all

    Regarding POSTing with LWP, it seems viable.

    bw, bliako

[reply]
[d/l]
      Thanks again, bliako.  I might try that sometime.  I seem to recall trying cookies last year when I was trying to get this working, but I should make sure and also try changing the user-agent as you've suggested.
[reply]

        for the record, what I suggested worked for me. What was important was the UA string, cookies-file did not make a difference and did not see them saved on disk. PhantomJS seems to keep session cookies on memory. So that angle is covered. You need to be explicit on UA though.

[reply]

In Section Seekers of Perl Wisdom