WWW::Mechanize redirection problem

eisdrache has asked for the wisdom of the Perl Monks concerning the following question:

I have website running javaserver pages that requires authentication via a login form on the front page.

If I access the page via a web browser I can submit my info and enter the site without a problem. However, when using WWW::Mechanize with the same exact login credentials I am denied entrance.

Here is the basic site setup:
login.jsp -> search.jsp -> mainIndex.jsp

When a user logs in the request is sent from login.jsp to search.jsp where the information is examined. If successful the request is redirected to the main page and the user is logged in, otherwise the request is redirected back to login.jsp with an error message asking to login to access the main page.

Here is a section of the code I am running:

my $mech = WWW::Mechanize->new();
$mech->cookie_jar(HTTP::Cookies->new());
$mech->get($target);
$mech->form_name('LoginForm');
$mech->field('loginId' => $login_id);
$mech->field('password' => $password);
$mech->click();
[download]

For whatever reason the search.jsp page does not like the request coming from WWW::Mechanize and I cannot figure out the problem, hence the reason for the post.

I realize this info is vague but I at least needed to create a starting point. Any help is greatly appreciated!

Comment on WWW::Mechanize redirection problem Select or Download Code

Replies are listed 'Best First'.
Re: WWW::Mechanize redirection problem by perrin (Chancellor) on Nov 28, 2006 at 06:27 UTC
The best I can give you with a vague question is a vague answer: try it in Firefox with the LiveHTTPHeaders extension and watch the traffic between the browser and the server. Adjust your mech script so that it does the same thing Firefox is doing. It's impossible for the JavaScript to do anything other than affect the HTTP exchange between browser and server, so everything that matters is contained in that exchange.	[reply]
Re^2: WWW::Mechanize redirection problem by eisdrache (Novice) on Nov 29, 2006 at 03:44 UTC
Thanks for the vague answer, it actually helped. I was previously unaware of the LiveHTTPHeaders extension, so after installing and using that I believe I found the problem. Apparently, the web request and the mechanize request were posting to two different pages. I checked the source code of the login page and the form has an action value of seach.jsp, however the submit button actually calls a javascript function which sends the request to login.jsp. Thanks again for your help!	[reply]
Re: WWW::Mechanize redirection problem by sanPerl (Friar) on Nov 28, 2006 at 09:04 UTC
You can try Win32::IE::Mechanize module. It will open IE's session. If your javascript is checking browser then you can fool it. Also check if, before accessing new page, 'sleep' for few seconds could do the job. Sometimes the script works too fast and you do not pull-out page till that time.	[reply]
Re: WWW::Mechanize redirection problem by former33t (Scribe) on Nov 28, 2006 at 12:47 UTC
I would think that sleeping for a second or two on the redirect might help. Also check for values that your browser may be POST'ing when the redirect is thrown. These are the two things that have screwed me up in the past with WWW::Mechanize.	[reply]
Re^2: WWW::Mechanize redirection problem by thinc (Novice) on Nov 28, 2006 at 22:03 UTC
I had problems with redirected posts as well. See LWP::UserAgent Docs. LWP::UserAgent does not follow redirects for posts by default, you have to tell it you want to do that. Since WWW::Mechanize is a subclass of LWP::UserAgent, you should be able to: `push @{ $mech->requests_redirectable }, 'POST';` [download]	[reply] [d/l]