WWW::Mechanize & <option>s

FrankRizzo has asked for the wisdom of the Perl Monks concerning the following question:

Guys, I'm trying to use mechanize to automate my checking of my local Subaru dealer's page. I would like to check his inventory of B9 Tribecas. In order to do that, I need to set the options for the search on his page. The section of HTML on his site looks like this:

<div class="qsFormElement" id="qsMake">
    <select name="make" onChange="myQuickSearch.setMake(this.options[t
+his.selectedIndex].value)">
        <option></option>
    </select>
</div>
<div class="qsFormElement" id="qsModel">
    <select name="model" onChange="myQuickSearch.setModel(this.options
+[this.selectedIndex].value)">
        <option></option>
    </select>
</div>
<div class="qsFormElement" id="qsTrim">
    <select name="trim" onChange="myQuickSearch.setTrim(this.options[t
+his.selectedIndex].value)">
        <option></option>
    </select>
</div>
[download]

Whenever I try to set any value in any of these fields, I get an "Illegal value" error from Mechanize. I also saw that there was some code that seems to populate the options, it looks like this:

//construct new QuickSearch object, passing inventory JSData object
var myQuickSearch = new QuickSearch(invJSData);

function initQuickSearch(){
    // this is to turn off sorting in the api; options will be display
+ed in JSDO order
    criteriaList[MODEL_NAME].setIsPreSorted(true);
    criteriaList[TRIM_NAME].setIsPreSorted(true);
    myQuickSearch.setShowNullSearchTypes(false);
    myQuickSearch.init(document.forms.quickSearchForm);
    // if there is no "new" inventory, the "New" option does not appea
+r in the dropdown.
    if (myQuickSearch.hasNew) setSearchType("new");
    else if (myQuickSearch.hasPreowned) setSearchType("preowned");    
}


/* Call in the onClick of the search dropdown */
setSearchType = function(newType) {
    // REQUIRED: this kicks off the dropdown population.
    myQuickSearch.setSearchType(newType);
    // by default it will load all makes; this is how you get it to lo
+ad models for just one make.
    if (newType == "preowned") myQuickSearch.setMake("Subaru");
}
[download]

Anyone know how I would automate this page? Thanks!

Comment on WWW::Mechanize & <option>s Select or Download Code

Replies are listed 'Best First'.
Re: WWW::Mechanize & <option>s by kwaping (Priest) on Mar 18, 2006 at 00:00 UTC
That's an interesting question. Here's what I would do, but it may not be the most elegant solution. I would load that page with Firefox, select the options that you want on the pulldowns, then select Page Info from the Tools menu. Click on the Forms tab and it should show you the names and current values of all form inputs. I would then build my WWW::Mechanize request using this information. It's a bit roundabout and not Perlish in any way, but it should do the trick. --- It's all fine and dandy until someone has to look at the code.	[reply]
Re^2: WWW::Mechanize & <option>s by FrankRizzo (Acolyte) on Mar 18, 2006 at 00:12 UTC
Great idea! (And I would have no problem doing... If it worked) :-D Here's the pertinent code: `$agent->submit_form( form_name => 'quickSearchForm', fields => { search => 'new', make => 'Subaru', model => 'B9 Tribeca', trim => '5-Passenger Ltd.', showImages => 'yes', fz => 'true', } );` [download] The field names, and values are exactly what show up in firefox, but it still gives me: `Illegal value 'new' at C:/Perl/site/lib/WWW/Mechanize.pm line 1178` [download]	[reply] [d/l] [select]
Re: WWW::Mechanize & <option>s by davidrw (Prior) on Mar 18, 2006 at 00:59 UTC
hmm.. looking briefly at source, that error comes from HTML::Form, but i didn't see a way to explicitly provide the possible_values ... At worst (assuming a GET is acceptable), instead of the `submit_form()` call you could just do (using a module to construct the query string if you like; needs escaping as well): `my %fields = ( search => 'new', make => 'Subaru', model => 'B9 Tribeca', trim => '5-Passenger Ltd.', showImages => 'yes', fz => 'true', ); $agent->get( $agent->current_form->agent . '?' . join("&", map { $_ ." +=".$fields{$_} } keys %fields) );` [download]	[reply] [d/l] [select]
Re: WWW::Mechanize & <option>s by Cody Pendant (Prior) on Mar 18, 2006 at 07:31 UTC
The way I would go about it is to use the `update_html` function to tweak the HTML currently held by Mechanize until it matches what the website expects, or at least contains the one value you need selected. That's what that function is there for after all. Mind you I'm a bit baffled as to what that JavaScript does, out of context, That setTrim function does something to the code of the select menu in the DOM as soon as you make a selection, and your snippet doesn't cover it. So, another FireFox solution, like kwapings only with one more layer -- you install the LiveHTTPHeaders extension, fill out the form and see what actual POST and/or GET values get sent by the browser, not what the form says? ($_='kkvvttuu bbooppuuiiffss qqffssmm iibbddllffss') =~y~b-v~a-z~s; print	[reply] [d/l]
Re^2: WWW::Mechanize & <option>s by FrankRizzo (Acolyte) on Mar 18, 2006 at 20:30 UTC
OK, stupid question. Once I get those values, I take it that I should populate the "fields" hash part of the "submit_form" with them?	[reply]


Just another Perl shrine
	PerlMonks