gnangia has asked for the wisdom of the Perl Monks concerning the following question:

Hello fello Monks,
I am using HTML::Parse to parse my html to do stuff. However, I have a requirement of submitting a form with user providing only the necessary details such as username/password and using the remaining of the url to extract all the form name/value pairs. This is because one of the name/value pairs is dynamic..i.e. the value is being generated everytime you get the login url before doing a post. As an example, this can be seen with http://my.yahoo.com. I know there is a module on CPAN called HTML::Form, but I am not sure how to use it in conjunction with HTML::Parse because the latter is indeed breaking the entire url apart. Can someone please provide me with a sample script of how I could go about parsing the form with HTML::Parse.
Thanks
  • Comment on Automating an HTML form submission with only required user info

Replies are listed 'Best First'.
Re: Automating an HTML form submission with only required user info
by Mr. Muskrat (Canon) on Dec 18, 2002 at 22:46 UTC
Re: Automating an HTML form submission with only required user info
by CountZero (Bishop) on Dec 18, 2002 at 22:43 UTC

    HTLM::FormParser seems the right module for this problem

    From the readme file: This module can be used to parse HTML text for forms. Each type of tag that is commonly found in a form (such as input, select, textarea etc) will trigger callbacks provided at the application level.

    The callback routines can then be used to cobble together the URL.

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

      HTML::Form is simpler to use for this sort of thing than HTML::FormParser (and comes with LWP, so it's probably more available).

      my $form=HTML::Form->parse($page, $url); $form->find_input('q1')->value($target); return $form->click;
      This will extract the first form from the HTML held in $page ($url is used to resolve relative URLs), and then q1 has its value set to $target. Finally, we return an HTTP::Request object (representing the form submission). If you want to turn this into a URL, then invoke uri_as_string on it.

      --
      Tommy
      Too stupid to live.
      Too stubborn to die.

Re: Automating an HTML form submission with only required user info
by PodMaster (Abbot) on Dec 18, 2002 at 23:09 UTC
    It is a losing battle you seek to fight, but anyways, let Mail::Webmail::Yahoo be your example ;)

    update: I'd like to point out that Simon Drabble, the author of Mail::Webmail::Yahoo is also the author of HTML::FormParser, which he uses in Mail::Webmail::Yahoo (along with his other modules) to parse forms, fill them appropriately, and send them back to yahoo .


    MJD says you can't just make shit up and expect the computer to know what you mean, retardo!
    ** The Third rule of perl club is a statement of fact: pod is sexy.

      I was looking at HTML::FormParser and noticed that at the end the module is calling $self->SUPER::parse($data). Can anyone please explain what does this mean? The only thing I can gather is that it is somehow calling HTML::Parser.