Processing page issue

pdilipm has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

I'm quite new to PERL. I'm trying to automate login to an application. Issue: Manually, once I enter the credentials and hit submit, it goes to a processing page and finally automatically goes to the homepage if login is valid. I'm trying to simulate this thru perl mechanize module. The form seems to get submitted, but the print response I get is the HTML of the processing page. My question is: how can I handle this processing page within the script?

This is what I've tried till now;

#! C:/Perl/lib/WWW
use WWW::Mechanize;
use strict;
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME}=0;

my $m = WWW::Mechanize->new(autocheck => 1);

my $url="https://rp6.bench.ehc.adp.com/siteminderagent/forms/LSPRH3/lo
+gin.fcc?TYPE=33554433&REALMOID=06-7d0449cd-e3ed-10fb-9f50-8498490a0cb
+3&GUID=&SMAUTHREASON=0&METHOD=GET&SMAGENTNAME=-SM-s2uV8pMx7pQ%2byUUrf
+iIGkme9FfRRMzZ8AkgpHMTZaaKvrgCvd%2fA99CuE2IG3sVvp&TARGET=-SM-https%3a
+%2f%2frp6%2ebench%2eehc%2eadp%2ecom%2findex_lsprh%2ehtml";

$m->get($url);
$m->success or die "login GET fail";

my $user='bdos0101-b01';
my $pass='bench_0';

my $login = $m->form_name("loginform");


$m->forms;
$m->form_name('loginform');
$login->value('USER' => $user);
$login->value('password' => $pass);

$m->submit();
$m->success or die "login POST fail";

print "Login done\n";
#print $m->content();
[download]

Here is the HTML of the processing page:

Login done
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=content-type content=text/html;charset=ISO-8859-1>
<META http-equiv=Refresh content=3;URL=/gamma/z2xcmnmainam/index>
<META   HTTP-EQUIV="Pragma" CONTENT="no-cache">
<META   HTTP-EQUIV="Expires" CONTENT=-1>
<META content="Nicolas Laigle" name=author>
<LINK media=screen href="index_img/style1.css" rel=stylesheet>
<LINK media=screen href="index_img/style2.css" rel=stylesheet>

<LINK media=screen href="index_img/style2.css"
rel=stylesheet><LINK media=screen
href="index_img/style2.css" rel=stylesheet>
<META content="MSHTML 6.00.2800.1106" name=GENERATOR></HEAD>
<BODY bgColor=#ffffff leftMargin=0 topMargin=0 marginwidth="0" marginh
+eight="0">

<TABLE cellSpacing=0 cellPadding=0 width="100%" border=0>
  <TBODY>
  <TR vAlign=top>
    <TD vAlign=bottom align=right>
      <TABLE cellSpacing=7 cellPadding=0 width="100%" border=0>
        <TBODY>
        <TR vAlign=bottom>
          <TD class=top noWrap
align=right></TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE>
<TABLE cellSpacing=14 cellPadding=0 width="100%" border=0>
  <TBODY>
  <TR vAlign=top>
    <TD align=center>
      <DIV class=h1>Connexion</DIV></TD>
    <TD align=center><IMG height=50 src="index_img/image1.gif"
      width=1 border=0></TD></TR>
  <TR>
    <TD colSpan=2>
      <DIV align=center><IMG height=20
      src="index_img/image1.gif" width=9 border=0>
      <TABLE cellSpacing=0 cellPadding=0 width="80%" border=0>
        <TBODY>
        <TR>
          <TD align=middle><IMG height=18 alt=height=24
            src="index_img/image2.gif" width=126
            border=0><BR><IMG height=30
            src="index_img/image1.gif" width=9 border=0>
            <P class=h4>Chargement en cours ...<BR></P><IMG
            height=30 src="index_img/image1.gif" width=9
            border=0><BR></TD></TR></TBODY></TABLE></DIV><BR></TD></TR
+>
  <TR>
    <TD colSpan=2></TD></TR></TBODY></TABLE>
[download]

Your suggestions/inputs will be greatly appreciated. Regards,

Comment on Processing page issue Select or Download Code

Replies are listed 'Best First'.
Re: Processing page issue by Corion (Patriarch) on May 29, 2012 at 10:51 UTC
The relevant part of the "processing" page is `<META http-equiv=Refresh content=3;URL=/gamma/z2xcmnmainam/index>` [download] (which is horribly malformed HTML). You'll have to parse that part out and then request the appropriate page. Also, consider learning how HTTP and HTML interact and how browsers usually behave.	[reply] [d/l]
Re^2: Processing page issue by pdilipm (Initiate) on May 30, 2012 at 10:18 UTC
Thank you for the tips. To start with, I hardcoded the URL to see if it works: `my $homepage = "https://xxx/gamma/z2xcmnmainam/index"; $m->get($homepage); print $m->content();` [download] This worked and I'm able to see the homepage! Now, I need to make it more generic for it to be able to work with any URL. I tried to use WWW::Mechanize::Plugin::FollowMetaRedirect $m->get($url); $m->follow_meta_redirect; But while trying to run, it gives me this error: Can't locate WWW/Mechanize/Pluggable.pm in @INC (@INC contains: C:/Dwimperl/perl /site/lib C:/Dwimperl/perl/vendor/lib C:/Dwimperl/perl/lib .) Could you please suggest how to fix this error? I also tried to parse the HTML for URL: use WWW::Mechanize; #use WWW::Mechanize::Plugin::FollowMetaRedirect; use strict; $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME}=0; my $m = WWW::Mechanize->new(autocheck => 1); $m->add_header( "Connection" => "keep-alive", "Keep-Alive" => "115 +"); my $url="https://rp6.bench.ehc.adp.com/siteminderagent/forms/LSPRH3/lo +gin.fcc?TYPE=33554433&REALMOID=06-7d0449cd-e3ed-10fb-9f50-8498490a0cb +3&GUID=&SMAUTHREASON=0&METHOD=GET&SMAGENTNAME=-SM-s2uV8pMx7pQ%2byUUrf +iIGkme9FfRRMzZ8AkgpHMTZaaKvrgCvd%2fA99CuE2IG3sVvp&TARGET=-SM-https%3a +%2f%2frp6%2ebench%2eehc%2eadp%2ecom%2findex_lsprh%2ehtml"; print "$INC{'WWW/Mechanize.pm'}\n"; $m->get($url); #$m->follow_meta_redirect; $m->success or die "login GET fail"; my $user='bdos0101-b01'; my $pass='bench_0'; my $login = $m->form_name("loginform"); $m->forms; $m->form_name('loginform'); $login->value('USER' => $user); $login->value('password' => $pass); $m->submit(); $m->success or die "login POST fail"; print "Login done\n"; #print $response->content(); if ($m =~ m!<meta\s+http-equiv=['"]refresh["']\s+content="\d+;([^"]+)" +!si) { warn "Refresh found (-> $1), following"; my $target = $1; $m->get($target); }; print $m->content(); [download] But it prints the processing page itself. Regards,	[reply] [d/l] [select]
Re^3: Processing page issue by Corion (Patriarch) on May 30, 2012 at 10:29 UTC
What part of the error message do you have problems with? `Can't locate WWW/Mechanize/Pluggable.pm in @INC (@INC contains: C:/Dwi +mperl/perl /site/lib C:/Dwimperl/perl/vendor/lib C:/Dwimperl/perl/lib + .` [download] How did you install WWW::Mechanize::Plugin::FollowMetaRedirect? Usually, Perl modules list all the other Perl modules they rely on and the Perl installation mechanisms follow these so-called pre-requisites and install them as well. If you did not use the installation mechanisms provided by Perl (`cpan`, `cpanp` or `cpanm`), please use them.	[reply] [d/l] [select]
Re: Processing page issue by Anonymous Monk on May 29, 2012 at 10:55 UTC
Here is a bag of clues :) follow LWP meta refresh, mechanize meta refresh ddg://site:perlmonks.org mechanize meta refresh -> WWW::Mechanize follow meta refreshes, Mechanize not recognizing link in <META HTTP-EQUIV="Refresh", www::mechanize reloading page mechanize meta -> WWW::Mechanize::Meta - Adds HEAD tag parsing to WWW::Mechanize, WWW::Mechanize::Plugin::FollowMetaRedirect - Follows 'meta refresh' link	[reply]