Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Clicking nameless buttons with WWW::Mechanize

by bobf (Monsignor)
on Mar 15, 2005 at 04:58 UTC ( #439544=perlquestion: print w/replies, xml ) Need Help??

bobf has asked for the wisdom of the Perl Monks concerning the following question:

I finally had an occasion to try out WWW::Mechanize, so I eagerly read the docs and started coding. I am somewhat embarassed to admit that I got stuck almost immediately. I am trying to log in, but I can't figure out how to post the form once I've filled it in because the submit button is in a table and it is unnamed.

The source HTML for this form is as follows:

<form method="POST" id="loginForm" action=" views%2Findex.jsp%3Fid%3Dhome.welcome" /> <table border="0" cellpadding="4" cellspacing="0"> <tr> <th align="right">Username:</th> <td align="left"><input type="text" name="username" tabindex="1" /></td> <td rowspan="2"> <a href="/mypharmgkb/profile/accountReminder.action"> Forgot your User ID and/or password?</a></td> </tr><tr> <th align="right">Password:</th> <td align="left"><input type="password" name="password" tabindex="2" /></td> </tr><tr> <td align="right">&nbsp;</td> <td align="left" colspan="3"><input type="submit" value="Sign In" tabindex="3" /> <input type="reset" tabindex="4" /></td> <td>&nbsp;</td> </tr> </table> </form>

This form is the second form on the page. I confirmed that I had the correct form by doing:

print Dumper( $mech_obj->current_form() );
Here is the relevant part of my code (meager, I know):
use strict; use warnings; use WWW::Mechanize; use HTTP::Cookies; my $mech_obj = WWW::Mechanize->new(); $mech_obj->get( $signin_url ); if( not $mech_obj->success() ) { print "Could not retrieve signin page:\n"; print $mech_obj->content(); die; } $mech_obj->form_number( 2 ); $mech_obj->set_fields( 'username' => $username, 'password' => $password );

I've tried to submit the form using the submit method, but it seems the first form on the page (a website search box) is submitted instead of the login form. I also tried using the click method, but I can't figure out what the name of the button is (using click without specifying the button name also submits the search box instead of the login form).

Since the button is in a table and doesn't appear to have a name, I thought using the optional arguments (x, y coordinates for the button) with the click method, but 1) the name of the button still appears to be required, and 2) I don't know how to determine the button coordinates on the page.

I Googled the net and Super Searched the Monastery's archives, but came up empty. Any ideas?

Thanks in advance!

Replies are listed 'Best First'.
Re: Clicking nameless buttons with WWW::Mechanize
by tphyahoo (Vicar) on Mar 15, 2005 at 12:24 UTC
    Longshot, but did you install Crypt::SSLeay? I noticed you're trying for an https url, and you can't do post requests with LWP without Crypt::SSLeay (which I think in turn requires Openssl, at least in my case - windows, ActiveState).

    This caused me no end of aggravation, because perl didn't give real good feedback that that was the problem, I had to dink around in Data::Dumper. Seems that a lot of people get bitten by this.

    So, just in case.

      No, I do not have Crypt::SSLeay installed - that never occurred to me. I am on Windows and I'm using ActiveState, so thanks for the Openssl comment, too.

      I'm starting to think I misinterpreted the error message. Thanks for an excellent suggestion!

      I'll update this node as soon as I try it.

      Update: I just got Openssl installed. I tried using PPM to install Crypt::SSLeay, but when I first installed perl I foolishly installed it in 'Program Files' and the Crypt::SSLeay installation choked on the space.

      Update: Intrepid brought to my attention the reloc_perl utility (included in ActiveState perl), which will relocate a perl installation to a new directory. I moved the installation out of the 'Program Files' directory and installed Crypt::SSLeay. However, I'm still getting an error when I try to submit the form:

      Unexpected field value at +(eval 5) line 1
      I'm baffled.

      Update: I tracked this error down. Please see the thread "Unexpected field value" error with WWW::Mechanize for more information.


        Well, I remember having trouble installing openssl as well, it was kind of confusing since there seems to be more than one distribution for windows. I finally got it working installing from ShiningLight Openssl.

        After installing that, check out

        And follow the tips there to do.....

        ppm install

        Gotta use uwinnipeg, since this isn't included in the standard activestate repositories, for reasons that probably have to do with exporting encryption or something, but I'm not sure.

        I think I tried making it myself with nmake, as well, and no dice. In the end it only worked with the above ppm install.

Re: Clicking nameless buttons with WWW::Mechanize
by PodMaster (Abbot) on Mar 15, 2005 at 07:38 UTC
    I've tried to submit the form using the submit method, but it seems the first form on the page (a website search box) is submitted instead of the login form.
    I sort of doubt that can happen
    use strict; use WWW::Mechanize; use WWW::Mechanize::FormFiller; use URI::URL; my $agent = WWW::Mechanize->new( autocheck => 1 ); my $formfiller = WWW::Mechanize::FormFiller->new(); $agent->env_proxy(); $agent->follow(''); $agent->get(''); $agent->form(1) if $agent->forms and scalar @{$agent->forms}; $agent->form_number(2); { local $^W; $agent->current_form->value('query', 'WWW::Mechanize'); + }; $agent->submit(); __END__ C:\>perl -MWWW::Mechanize -e"die WWW::Mechanize->VERSION" 1.11_01 at -e line 1. C:\>perl -MWWW::Mechanize::Shell -e shell Module File::Modified not found. Automatic reloading disabled. SetConsoleMode failed, LastError=|6| at C:/Perl/site/lib/Term/ReadKey. +pm line 268. >open Use of uninitialized value in concatenation (.) or string at C:/Perl/s +ite/lib/WWW/ line 1743, <IN> line 1. Can't find any link matching on this page () at C +:/Perl/site/lib/WWW/Mechanize/ line 1028 Can't call method "code" on an undefined value at C:/Perl/site/lib/WWW +/Mechanize/ line 1034, <IN> line 1. >get Retrieving>>>forms Form [1] GET query= (text) <NONAME>= Go (submit) Form [2] GET mode=module (option) [*module/Module|dist/Distr +ibution|author/Author|doc/Documentation] query= (text) <NONAME>=Search (submit) Form [3] POST unickname= (text) returnto=// (hidden readonly) op=userlogin (hidden readonly) upasswd= (password) login_temp=<UNDEF> (checkbox) [*<UNDEF>/off|yes/Public T +erminal] userlogin=Log in (submit)>form 2 GET mode=module (option) [*module/Module|dist/Distr +ibution|author/Author|doc/Documentation] query= (text) <NONAME>=Search (submit)>value query WWW::Mechanize>submit 200>>>scr +ipt

    MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
    I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
    ** The third rule of perl club is a statement of fact: pod is sexy.

Re: Clicking nameless buttons with WWW::Mechanize
by jbrugger (Parson) on Mar 15, 2005 at 05:12 UTC
    I'm a bit confused what you exacly mean, please correct me if i'm wrong. You have 2 forms in one page, and want the submit button to submit the proper form?
    When using:
    <form action="javascript:alert('1')"> <input type=submit /> </form> <form action="javascript:alert('2')"> <input type=submit /> </form>
    It seems to work fine for me, i just can have a different action per form, and the button submits the form it is in.
    Ah i see...
    $mech->submit_form( form_number => 1, fields => { username => 'mungo', password => 'lost-and-alone', } ); # as from $mech->submit_form( ... ) # If button is not passed, then the submit() method is used instea +d.
    Its described here

      Thanks for the reply. At first I thought that would work, too. Specifically, I tried:

      $mech_obj->submit_form( form_number => 2, fields => { 'username' => $username, 'password' => $password } );

      But I got the same result as when I used the submit method directly: the search box (form 1) was submitted instead (despite form 2 being selected). Thanks for trying, but I'm still looking for a solution.

Re: Clicking nameless buttons with WWW::Mechanize
by Cody Pendant (Prior) on Mar 15, 2005 at 10:03 UTC
    I just tried logging in to which is presumably what you're trying to do?

    I don't have a username and password, but this code worked just fine (it gave me an error because it's an HTTPS form action and I don't have the right module, but it definitely didn't submit the search form):

    use WWW::Mechanize; my $agent = WWW::Mechanize->new(); $agent->get(''); $agent->form(2); $agent->field('username','perlmonk'); $agent->field('password','knomlrep'); $agent->submit();

    Obviously my code's slightly different to yours, but I'm baffled as to how you've submitted the other form.

    =~y~b-v~a-z~s; print
Re: Clicking nameless buttons with WWW::Mechanize
by mdillon (Priest) on Mar 15, 2005 at 06:16 UTC
    It looks like you can pass a value parameter to click_button. If the submit buttons have different values, that should work.

    Taking another tack, how about adding a name to each button? I suppose there's probably a reason they don't have names, but that would obviously simplify things. Sorry if this suggestion sounds dismissive.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://439544]
Front-paged by monkfan
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2022-12-04 22:45 GMT
Find Nodes?
    Voting Booth?

    No recent polls found