rembrantxvii has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to get WWW::Mechanize to find a button on a page and click it, but the click_button() function doesn't seem to work properly. For a test example, consider the following website: 
<html>
<form>
<button type="submit" id="test" name="test">My Button</button>
</form>
</html>

[download]
My mechanize script then goes and grabs the webpage: 
use WWW::Mechanize;
use Data::Dumper;

# URL to retrieve
$url = 'http://myurl.html ';

# Create an instance of Mechanize to scrape
my $mechanize= WWW::Mechanize->new( autocheck => 1 );

# Grab the form
$mechanize->get($url);
$mechanize->form_number('1');

($output) = $mechanize->current_form();
    open(FH, '>dump.txt');
    print FH Dumper($output);
    close(FH);
   
$mechanize->click_button(name => 'test');

[download]
Mechanize then gives an error 'no clickable input with name test'. Dumper shows the form was located correctly, but that it contained no items.

However, if I modify that original HTML code snippet to say <input type="submit></input> rather than <button>, Mechanize will successfully run the click_button() routine and Dumper shows that the form contained an HTML::Form::SubmitInput object.

How can I click on buttons then that are of the form <button type="submit"> or even find those buttons on the page? From the dump, it seems like Mechanize ignores them all together. Am I mistaking the functionality of click_button()? Click() may work for a single case, but when there are multiple buttons on a page, I need click_button().

Any help is appreciated.

-- RembrantXVII

Replies are listed 'Best First'.
Re: WWW::Mechanize click_button
by Limbic~Region (Chancellor) on Jan 23, 2008 at 20:23 UTC
    rembrantxvii,
    In my 2 minute dive of the source, WWW::Mechanize uses HTML::Form. While HTML::Form recognizes and parses button tags, it defaults to IgnoreInput action with a sub value { return }

    I could be way off base here as I do not know the internals of these modules at all.

    Cheers - L~R

[reply]
[d/l]
Re: WWW::Mechanize click_button
by talexb (Chancellor) on Jan 23, 2008 at 20:59 UTC

    I would suggest you try using $mech->submit() instead of the click_button method.

    Alex / talexb / Toronto

    "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

[reply]
[d/l]
[select]
      If I might add a bit of HTML wisdom... submit buttons look like buttons (to humans) but they are a different type of objects. One should really consider them like "submits" and not like "buttons". If you have free time to spare, I recommend reading about the HTML DOM. It's a useful read if you intend to work more with html/web pages.

      Even though in HTML the submit gets click-ed before the event bubbles up to the form that gets submit-ted, what you really are trying to do is submit the form. So it doesn't matter that much to you if the submit gets clicked as long as the form is submitted, right?
[reply]
Re: WWW::Mechanize click_button
by amarquis (Curate) on Jan 23, 2008 at 20:17 UTC

    That's interesting. Does Mechanize support Javascript? I looked through the docs quickly and didn't see anything.

    The button element has been around for quite some time, the only reason I could think of for not supporting clicking them is that in practical use their clicks are grabbed by a Javascript event handler and processed that way. I see buttons of type submit fairly rarely.

    Specific to your question: I tried to get Mechanize to click using the "click" method, supplying the name of the button, but that didn't work either. Is it possible, in your situation, to cheat and do a substitution of submit buttons to submit inputs?

    Edit: L~R's source dive makes sense, the best solution I can think of is still to cheat and sub out the button for an input before Mechanize does the handling. (Oh, and maybe submit a bug against HTML::Form, asking for button type="submit" tags to be handled?)

[reply]
        That's interesting. Does Mechanize support Javascript? I looked through the docs quickly and didn't see anything.

      Then you didn't read the WWW::Mechanize::FAQ -- it's probably the most amusing Perl FAQ I've ever read. And apparently people *still* write to Andy asking about JavaScript support. I managed to squirm around that challenge by using Test::WWW:Mechanize on my CGI::Application web application in which I use a few bits of JavaScript -- mostly to pop-up alerts that say "Are you sure?" to confirm various delete operations.

      Seems to work fine.

      Alex / talexb / Toronto

      "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

[reply]
        talexb,
        When I mentioned WWW::Mechanize::Plugin::JavaScript to Andy the other night on #perl (freenode), he was unaware of its existance. Hopefully the module author will continue to work on it because it would make a lot of people happy.

        Cheers - L~R

[reply]
      amarquis,
      Oh, and maybe submit a bug against HTML::Form, asking for button...

      Actually, I just looked into that too. It seems that bug was filed 2 years ago.

      Cheers - L~R

[reply]

        Seriously, man, stop bringing in your facts and hard information, it's making my idle speculation look bad!

        :)

[reply]
Re: WWW::Mechanize click_button
by Popcorn Dave (Abbot) on Jan 23, 2008 at 20:15 UTC
    Just a thought here, but have you actually looked at the source of the page you're trying to do this with? Is the button contained in either a frame or a DIV tag?

    Revolution. Today, 3 O'Clock. Meet behind the monkey bars.

    I would love to change the world, but they won't give me the source code

[reply]
Re: WWW::Mechanize click_button
by roweco (Initiate) on Jan 24, 2008 at 14:38 UTC
    If you are on Windows then Win32::IE::Mechanize can work where the "Mech" can't (e.g javascript hurdles). It can help to view frame info with something like Firefox's Web Developer add-on to work out form info.
    Example code ... 
    use Win32::IE::Mechanize;
my $mech = Win32::IE::Mechanize->new(visible => 1);
# Connect to server
$mech->get( "http://web_server:8080" );
$mech->success or die $mech->response->status_line;

my $html = $mech->content;

# Log into server
$mech->form_name("loginform");
$mech->field('userName', 'courtenay@db.com');
$mech->field('loginPassword', '');
$mech->click_button( value => ' Login ' );

$html = $mech->content;

    [download]
    ...etc etc
[reply]
[d/l]

Back to Seekers of Perl Wisdom