Click button not working in WWW::Mechanize

monkfan has asked for the wisdom of the Perl Monks concerning the following question:

On this website. I am attempting to do these two steps:

Set "Start<-ATG" into -450 and "ATG->END" 50, as the value
Click on "Retrieve Sequence" button.

But how come the following script I have doesn't do the job? Especially the "click_button" method.

use WWW::Mechanize;   
my $mech = WWW::Mechanize->new();

my $tf_add = "http://rulai.cshl.edu/cgi-bin/SCPD/getgene2?ARG1";
$mech->get($tf_add); 


$mech->set_fields( 'start' => -450, 'end' => 50 ); # this part works

# but not this:

$mech->click_button( 
        name   => "action",       
        value  => "Retrieve sequence"
);

$mech->submit();

my $result = $mech->content();
print "$result\n";
[download]

The snippet of the HTML structure in the related website can be found here :

<html><title>ARG1</title>
<body bgcolor=white>
<h2 align=center>ARG1</h2><hr>
<form method="post" action="/cgi-bin/SCPD/getgene2?ARG1" enctype="appl
+ication/x-www-form-urlencoded">
<input type="submit" name="action" value="Get mapped sites" /><input t
+ype="submit" name="action" value="Get putative sites" /><input type="
+submit" name="action" value="Get intergenic region" /><br /><input ty
+pe="submit" name="action" value="Retrieve sequence" />Start<-ATG <inp
+ut type="text" name="start" value="-500" size="5" maxlength="5" />ATG
+->End <input type="text" name="end" value="50" size="5" maxlength="5"
+ /><div></div></form><hr>
<pre>
       ORF     YOL058W
      GENE     ARG1
        XX
    FACTOR     <a href= 
                "getfactor?ARC">ARC</a>
COORDINATE     (-269, -247)
  SEQUENCE     CATTTAAAGGCAGGAGAGAGAGA
 REFERENCE     Yeast 1995, 11:1367-1380

.... skip some entries...

</pre>
[download]

And the desired final output looks like this (by hand clicking).

<html><title>ARG1</title>
<body bgcolor=white>
<h2 align=center>ARG1</h2><hr>
<form method="post" action="/cgi-bin/SCPD/getgene2?ARG1" enctype="appl
+ication/x-www-form-urlencoded">
<input type="submit" name="action" value="Get mapped sites" /><input t
+ype="submit" name="action" value="Get putative sites" /><input type="
+submit" name="action" value="Get intergenic region" /><br /><input ty
+pe="submit" name="action" value="Retrieve sequence" />Start<-ATG <inp
+ut type="text" name="start" value="-450" size="5" maxlength="5" />ATG
+->End <input type="text" name="end" value="50" size="5" maxlength="5"
+ /><div></div></form><hr>
<pre>
>YOL058W  ARG1  218759  219259
CGAGCTTTTTCACTGCAGTAATTCTCCACATGGGCCCAGCCACTGAGATA
AGAGCGCTATGTTAGTCACTACTGACGGCTCTCCAGTCATTTATGTGATT
TTTTAGTGACTCATGTCGCATTTGGCCCGTTTTTTTCCGCTGTCGCAACC
TATTTCCATTAACGGTGCCGTATGGAAGAGTCATTTAAAGGCAGGAGAGA
GAGATTACTCATCTTCATTGGATCAGATTGATGACTGCGTACGGCAGATA
GTGTAATCTGAGCAGTTGCGAGACCCAGACTGGCACTGTCTCAATAGTAT
ATTAATGGGCATACATTCGTACTCCCTTGTTCTTGCCCACAGTTCTCTCT
CTCTTTACTTCTTGTATCTTGTCTCCCCATTGTGCAGCGATAAGGAACAT
TGTTCTAATATACACGGATACAAAAGAAATACACATAATTGCATAAAATA
ATGTCTAAGGGAAAAGTTTGTTTGGCTTATTCTGGTGGTTTAGATACCTC
[download]

Finally I would just like to extract the DNA sequence after <pre> tag above, starting from ">".

Regards,
Edward

Comment on Click button not working in WWW::Mechanize Select or Download Code

Replies are listed 'Best First'.
Re: Click button not working in WWW::Mechanize by wfsp (Abbot) on Sep 21, 2006 at 10:22 UTC
I commented out this line: `#$mech->submit();` [download] and got: ---------- Capture Output ---------- > "c:\perl\bin\perl.exe" _new.pl <html><title>ARG1</title> <body bgcolor=white> <h2 align=center>ARG1</h2><hr> <form method="post" action="/cgi-bin/SCPD/getgene2?ARG1" enctype="appl +ication/x-www-form-urlencoded"> <input type="submit" name="action" value="Get mapped sites" /><input t +ype="submit" name="action" value="Get putative sites" /><input type=" +submit" name="action" value="Get intergenic region" /><br /><input ty +pe="submit" name="action" value="Retrieve sequence" />Start<-ATG <inp +ut type="text" name="start" value="-450" size="5" maxlength="5" />ATG +->End <input type="text" name="end" value="50" size="5" maxlength="5" + /><div></div></form><hr> <pre> >YOL058W ARG1 218759 219259 CGAGCTTTTTCACTGCAGTAATTCTCCACATGGGCCCAGCCACTGAGATA AGAGCGCTATGTTAGTCACTACTGACGGCTCTCCAGTCATTTATGTGATT TTTTAGTGACTCATGTCGCATTTGGCCCGTTTTTTTCCGCTGTCGCAACC TATTTCCATTAACGGTGCCGTATGGAAGAGTCATTTAAAGGCAGGAGAGA GAGATTACTCATCTTCATTGGATCAGATTGATGACTGCGTACGGCAGATA GTGTAATCTGAGCAGTTGCGAGACCCAGACTGGCACTGTCTCAATAGTAT ATTAATGGGCATACATTCGTACTCCCTTGTTCTTGCCCACAGTTCTCTCT CTCTTTACTTCTTGTATCTTGTCTCCCCATTGTGCAGCGATAAGGAACAT TGTTCTAATATACACGGATACAAAAGAAATACACATAATTGCATAAAATA ATGTCTAAGGGAAAAGTTTGTTTGGCTTATTCTGGTGGTTTAGATACCTC > Terminated with exit code 0. [download] From the docs: $mech->submit() Submits the page, without specifying a button to click. Actually, no button is clicked at all.	[reply] [d/l] [select]
Re^2: Click button not working in WWW::Mechanize by mbarbosa (Initiate) on Sep 21, 2006 at 10:25 UTC
yup, monkfan was submitting the form twice and collecting the content only after the submit that did not matter.	[reply]
Re^2: Click button not working in WWW::Mechanize by monkfan (Curate) on Sep 21, 2006 at 12:05 UTC
Strange wfsp, I tried to comment out "$mech->submit()" but it still does not give the result as you get (i.e. same old output). Did you use the exact code I have in my OP? BTW I am running under Linux, could it bet the case? `Linux myhost.com.sg 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006 + i686 i686 i386 GNU/Linux` [download] Regards, Edward	[reply] [d/l]
Re^3: Click button not working in WWW::Mechanize by wfsp (Abbot) on Sep 21, 2006 at 12:11 UTC
Sorry, forgot to mention that I also removed: `name => "action",` [download] because all the buttons had that name. `#!/usr/bin/perl use warnings; use strict; use WWW::Mechanize; my $mech = WWW::Mechanize->new(); my $tf_add = "http://rulai.cshl.edu/cgi-bin/SCPD/getgene2?ARG1"; $mech->get($tf_add); $mech->set_fields( 'start' => -450, 'end' => 50 ); # this part works # but not this: $mech->click_button( value => "Retrieve sequence" ); #$mech->submit(); my $result = $mech->content(); print "$result\n";` [download]	[reply] [d/l] [select]
Re: Click button not working in WWW::Mechanize by Anonymous Monk on Sep 21, 2006 at 08:53 UTC
`my $response = $mech->click_button( name => "action", value => "Retrieve sequence" ); die "click_button should have worked: ", $response->status_line;` [download]	[reply] [d/l]
Re^2: Click button not working in WWW::Mechanize by monkfan (Curate) on Sep 21, 2006 at 08:59 UTC
Hi, Thanks for the reply. After I tried your snippet above. It shows this: `click_button should have worked: 200 OK at test.pl line 25.` [download] And it still doesn't print the desired output as shown above? Any reason? Hope to hear from you again. Regards, Edward	[reply] [d/l]