Submitting form data to Javascript form

gebelo has asked for the wisdom of the Perl Monks concerning the following question:

I'm attempting my first complicated scraping project and have hit a wall. To get at the data I want to scrape, I need to submit several forms first. I'm using www:mech and got through the first form easy:

$browser->get( $starting_url );
$browser->form_name("GS");
$browser->submit();
[download]

This takes me to a new page with another form -- but this one has a Javascript "onClick" routine, and when I try various permutations of the following, it doesn't accept my input: This is the form code:

<form name="gs" METHOD="post" action="" onSubmit="cntyzip()" >
  <table width=70% align="center">
    <tr>
      <td align=center colspan=2> You must either choose a <b>County <
+font
        color="#FF0000"> &nbsp;or&nbsp;</font></b> enter a <b>Zip Code
+</b></td></tr>
     <tr><td>&nbsp;</td></tr>
    <tr>
      <td colspan="2"> <table width = 76% align="center">
          <tr>
            <td width="16%" align=left><font size=2>County:</font></td
+>
            <td width="30%" align=left><font size=2>
              <select name="county">
                <option value="">Choose a county</option>
                <option value="01ATLANTIC">ATLANTIC</option>
[download]

And these are two of the methods I've attempted to get the thing to go:

$browser->form_name( "gs" );
   $browser->set_fields(
       county => '01ATLANTIC'
       
   );
   $browser->submit();

# $browser->form_name("gs");
# $browser->field( "county", "01ATLANTIC" );
#  $browser->cntyzip();
[download]

I've tried this in both regular mech and the Win32::ie version, but no luck... Any advice?

Comment on Submitting form data to Javascript form Select or Download Code

Replies are listed 'Best First'.
Re: Submitting form data to Javascript form by Kanji (Parson) on Mar 25, 2005 at 19:52 UTC
Find out what cntyzip() actually does and rewrite your HTML as appropriate before form submitting. `$browser->get( $starting_url ); ## Reverse engineer cntryzip() here. my $html = $brwoser->content; $html =~ s/" onSubmit="cntyzip()"/cntyzip.cgi"/isg; ## Insert de-JS'd version here. $browser->update_html($html); $browser->form_name("GS"); $browser->submit();` [download] Update: s/mech/browser/; # oops --k.	[reply] [d/l]
Re^2: Submitting form data to Javascript form by gebelo (Initiate) on Mar 28, 2005 at 21:09 UTC
This is the solution I came up with, with the help of a friend.. it basically re-writes the form HTML and bypasses the Java script In this case, the original search was by county and $line = the value I pulled in from a list... `my $form = $browser->form_name( 'formname' ); $form->action( 'https://destinationurl' ); $form->value( county => $line ); $browser->request( $form->click( 'buttonname' ) );` [download]	[reply] [d/l]