in reply to redirection issue with WWW::Mechanize

I had to do the following in a test script to get Mechanize to do what I wanted: 

# turn off following redirects
$mech->requests_redirectable( [] );

[download]
You would then probably need to get the redirect URL from the response and issue another request to that.

That, or maybe set the redirect limit to 1 or look into $mech->redirect_ok().

It helps to remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.

Replies are listed 'Best First'.
Re^2: redirection issue with WWW::Mechanize
by Special_K (Pilgrim) on May 20, 2014 at 07:04 UTC
    Is there a WWW::Mechanize function that accesses the redirect URL directly from the response, or do I need to parse through the entire $post_response->decoded_content() to find it?
[reply]
[reply]
      OK, here is what I did:

      First at the beginning of my program I have: 

      $agent->requests_redirectable( [] );

      [download]


      Now here is where I click a continue button that loads the page that is supposed to redirect me to the final page (note that redirects are disabled and I am retrieving and then GETing the redirect URL manually): 

      $post_response = $agent->click('continue');
my $status = $agent->status();

if (($status >= 300) && ($status < 400)) 
{
    my $location = $agent->response()->header('Location');
    
      if (defined $location) 
      {
        print "Redirected to $location\n";
        $post_response = $agent->get(URI->new_abs($location, $agent->b
+ase()));
        printf("post_response is %s\n", $post_response->status_line);
      }
}

      [download]
      This seems to successfully capture the redirect URL and outputs the following (actual long hex strings and other info replaced with 'X'): 
      Redirected to /misc/XXXX/processocp.cgi?email=XXXX&back=httpXXXXX&spla
+sh=X&vskey=XXXX
post_response is 200 OK

      [download]
      However I can tell from my program that is isn't successfully navigating to the final screen of the process, because when I run: 
      $post_response->decoded_content()

      [download]
      I get (again, the info in the redirect URL has been replaced with 'X'): 
      <!DOCTYPE html>
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="content-type">
<meta http-equiv="refresh" content="1;url=http://XXXX/processocp.cgi?e
+mail=XXXXX&back=XXXXXX&vskey=XXXXX">
</head>
<body>
<div style="font-size: 1.5em; text-align: center; width: 300px; margin
+: 0px auto; margin-top: 40px">
Processing...please wait<br/>
</div>
</body>
</html>

      [download]


      As you can see, it still seems to be stuck at the "processing.." screen, even though I told the program to "get" the redirect URL, i.e. the URL that would normally be automatically retrieved if redirects were enabled.

      What is going on here?
[reply]
[d/l]
[select]
        OK, I solved it. The following does work: 

        $agent->follow_meta_redirect;

        [download]


        but only if you do NOT also have this in your code: 

        $agent->requests_redirectable( [] );

        [download]


        Alternately, you can also just parse the content returned by: 

        $post_response->decoded_content()

        [download]


        when the page containing the redirect is first loaded, and get the redirect URL from that. Again, the key is that you must NOT have: 

        $agent->requests_redirectable( [] );

        [download]


        in your code. I think it's strange that if you enable redirects you still must manually retrieve the redirect URL, and if you disable redirects then even manually retrieving the URL doesn't work.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom