Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello!
There is a website which has a page that I'm trying to get to programatically. There is a process that needs to be followed to get to the desired page. Now I've already found short cuts to get to the second-to-last page in the process, the problem is that this last page has a form. I'm trying to simulate the action of clicking the "Submit" button.

Now the kicker, if I take all the form variables and put them into the URL http://foo.com/bar?baz=true, etc. I get dumped back to the first page in the process. So what I need is a way to do simulate a person using a web-browser clicking on submit.
If I'm being too vague I'm sure somebody will tell me.

Any useful help is appreciated!

Replies are listed 'Best First'.
(jeffa) Re: Help with forms
by jeffa (Bishop) on Jan 17, 2001 at 01:28 UTC
    Too vague? Nonesense!

    I'm trying to simulate the action of clicking the "Submit" button

    Simple: use LWP::Simple

    say you have a CGI script like this (called foo.cgi):

    #!/usr/bin/perl use strict; use CGI; my $cgi = new CGI; print $cgi->header; if (my $foo = $cgi->param('foo')) { print "foo = $foo"; } else { print "nothing special"; }
    all this script does is check for a parameter named 'foo' - if found, print it's value, otherwise print some default message.

    If you create an HTML document that contains a form that points to this script, and include a text box named 'foo' and a submit button - then when the user clicks the submit button, the value of 'foo' is sent to the script.

    So, to automate this behaviour in a Perl script, you can either roll out your own socket code, or be cool and use the LWP::Simple CPAN module - which by the way, is not the only module for such a purpose (check out LWP::UserAgent for more advanced control).

    Here is the simple web bot:

    #!/usr/bin/perl -w use strict; use LWP::Simple; print "with no arguments: "; print get('http://localhost/cgi-bin/foo.cgi'), "\n"; print "with arguments: "; print get('http://localhost/cgi-bin/foo.cgi?foo=bar'), "\n";
    And that's it.

    Jeff

    L-LL-L--L-LL-L--L-LL-L--
    -R--R-RR-R--R-RR-R--R-RR
    F--F--F--F--F--F--F--F--
    (the triplet paradiddle)
    
Re: Help with forms
by chromatic (Archbishop) on Jan 17, 2001 at 01:32 UTC
    Submit buttons usually pass a parameter as well. Some scripts check for this. You may need to pass the name and value associated with the submit button.

    If nothing's specified in the HTML, nothing's passed (at least as I understand the specification).

    It's worth checking, though.

Re: Help with forms
by little (Curate) on Jan 16, 2001 at 23:54 UTC
    What does the form say as a method? post or get ?
    Have a nice day
    All decision is left to your taste
Re: Help with forms
by EvanK (Chaplain) on Jan 16, 2001 at 23:59 UTC
    If the http://url.of/the?script doesnt work, the form might be designed to specifically use the POST method...try using a POST request.

    ______________________________________________
    When I get a little money, I buy books. If I have any left over, I buy food and clothes. -Erasmus

Re: Help with forms
by ichimunki (Priest) on Jan 17, 2001 at 00:00 UTC
    Yes. You are being too vague. Please post some sample code so we can see how you are building your HTTP::Request object. :)

    And a snippet of the source for the form would be helpful as well.
A reply falls below the community's threshold of quality. You may see it by logging in.