Special_K has asked for the wisdom of the Perl Monks concerning the following question:

I am using the WWW::Mechanize module to automate a login/interaction with a website. I have a section of code that is to be executed on each page that I navigate through.

On one of the first pages, I enter a PIN and click a "continue" button. That takes me to a screen that says "verifying PIN", which then automatically redirects to the next page that accepts user input.

The problem I am having is that I don't know how to tell my script to "wait" for the PIN verification page to redirect me to the next page that requires user input. Consequently, my script is trying to enter data on the "verifying PIN" page that should actually be entered on the next page.

How can I tell my script to wait for the verification page to redirect to the next page that requires user input?

UPDATE: I navigated through the page using Firefox with javascript disabled and the redirection still went through properly, so javascript does not seem to be required to make the redirect occur, meaning I should be able to use WWW::Mechanize, correct?

Replies are listed 'Best First'.
Re: redirection issue with WWW::Mechanize
by boftx (Deacon) on May 18, 2014 at 21:26 UTC

    I had to do the following in a test script to get Mechanize to do what I wanted: 

    # turn off following redirects
$mech->requests_redirectable( [] );

    [download]
    You would then probably need to get the redirect URL from the response and issue another request to that.

    That, or maybe set the redirect limit to 1 or look into $mech->redirect_ok().

    It helps to remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.
[reply]
[d/l]
[select]
      Is there a WWW::Mechanize function that accesses the redirect URL directly from the response, or do I need to parse through the entire $post_response->decoded_content() to find it?
[reply]
[reply]
        OK, here is what I did:

        First at the beginning of my program I have: 

        $agent->requests_redirectable( [] );

        [download]


        Now here is where I click a continue button that loads the page that is supposed to redirect me to the final page (note that redirects are disabled and I am retrieving and then GETing the redirect URL manually): 

        $post_response = $agent->click('continue');
my $status = $agent->status();

if (($status >= 300) && ($status < 400)) 
{
    my $location = $agent->response()->header('Location');
    
      if (defined $location) 
      {
        print "Redirected to $location\n";
        $post_response = $agent->get(URI->new_abs($location, $agent->b
+ase()));
        printf("post_response is %s\n", $post_response->status_line);
      }
}

        [download]
        This seems to successfully capture the redirect URL and outputs the following (actual long hex strings and other info replaced with 'X'): 
        Redirected to /misc/XXXX/processocp.cgi?email=XXXX&back=httpXXXXX&spla
+sh=X&vskey=XXXX
post_response is 200 OK

        [download]
        However I can tell from my program that is isn't successfully navigating to the final screen of the process, because when I run: 
        $post_response->decoded_content()

        [download]
        I get (again, the info in the redirect URL has been replaced with 'X'): 
        <!DOCTYPE html>
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="content-type">
<meta http-equiv="refresh" content="1;url=http://XXXX/processocp.cgi?e
+mail=XXXXX&back=XXXXXX&vskey=XXXXX">
</head>
<body>
<div style="font-size: 1.5em; text-align: center; width: 300px; margin
+: 0px auto; margin-top: 40px">
Processing...please wait<br/>
</div>
</body>
</html>

        [download]


        As you can see, it still seems to be stuck at the "processing.." screen, even though I told the program to "get" the redirect URL, i.e. the URL that would normally be automatically retrieved if redirects were enabled.

        What is going on here?
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom