scraping usps

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

USPS has a shipping calculator that I'd like to use. The problem is, they are using JavaScript for the package sizes on the first page. I have NO idea how to get through the JavaScript with WWW::Mech.

Can anyone show an example? I know how to use this module but not for something I can't get a name for that's nested with JS.

Comment on scraping usps

Replies are listed 'Best First'.
Re: scraping usps by bmann (Priest) on Nov 23, 2005 at 02:25 UTC
Here's an answer to what I think is your underlying question, how to automate retrieval of USPS shipping rates. The USPS has published a set of tools at http://www.usps.com/webtools/. Registration and use of these tools are free, and the post office wants people to use this service. Once you've signed up, have a look at Business::Shipping. Good luck!	[reply]
Re: scraping usps by brian_d_foy (Abbot) on Nov 23, 2005 at 07:42 UTC
They are using some Javascript, but you don't need it for the form. On http://www.usps.com/tools/calculatepostage/welcome.htm?from=home&page=0061calculatepostage, the pounds input is named "P" and the ounces input is named "O". The package types are a radio button group named "MT". Perhaps you're talking about something else? If you can use their web API, do that. They are pretty slow to respond with useable developer keys though, so you might have to stick with WWW::Mechanize for a bit. -- brian d foy <brian@stonehenge.com> Subscribe to The Perl Review	[reply]
Re: scraping usps by swares (Monk) on Nov 23, 2005 at 07:24 UTC
If you still want to know how to find that out you could use a caching proxy server like burp or the Tamper Data Extension for Firefox to intercept and view what is going to be posted.	[reply]