ttcuberat has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,

I'll be going on vacation soon and am trying to set up a script to ocassionaly get me a airfare quote. I've set up form params according to the page, but I cannot get past the second page in the process. I keep getting an error from the page saying that I incorrectly passed a value from the first page. Here's my code. Any help would be appreciated.

#!/usr/local/bin/perl use HTTP::Request::Common qw(POST); use HTTP::Request; use LWP::UserAgent; use HTTP::Cookies; use LWP::Debug '+'; use strict; my $DEBUG = 1; my $VERBOSE = 1; my $ua = LWP::UserAgent->new; $ua->agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)'); my $cookie_jar = HTTP::Cookies->new(); $ua->cookie_jar($cookie_jar); my $req = new HTTP::Request ( 'GET', 'http://www.aa.com' ); my $res = $ua->request($req); $cookie_jar->extract_cookies($res); my $content = $res->content; my $session_id = $content; $session_id =~ s/.*(jsessionid=.*?)".*/$1/gis; my $submit_url = "http://www.aa.com/apps/reservations/FlightSearch +Results.jhtml;$session_id"; my $ua = new LWP::UserAgent; $req = POST $submit_url, [ '_DARGS' => '/apps/modules/makeReservation.jhtml +.8', 'currentCalForm' => 'dep', 'currentCodeForm' => '', 'tripType' => 'roundTrip', '_D:tripType' => ' ', 'origin' => 'DFW', '_D:origin' => ' ', 'departureMonth' => '11', '_D:departureMonth' => ' ', 'departureDay' => '24', '_D:departureDay' => ' ', 'departureTime' => '1', '_D:departureTime' => ' ', 'departureYear' => '', 'destination' => 'LGA', '_D:destination' => ' ', 'returnMonth' => '12', '_D:returnMonth' => ' ', 'returnDay' => '3', '_D:returnDay' => ' ', 'returnTime' => '1', '_D:returnTime' => ' ', 'returnYear' => '', 'numAdultPassengers' => '2', '_D:numAdultPassengers' => ' ', 'numChildPassengers' => '0', '_D:numChildPassengers' => ' ', 'searchType' => 'schedule', '_D:searchType' => '', '/aa/travelinfo/SearchFlightsFormHandler.submit' => '', '_D:/aa/travelinfo/SearchFlightsFormHandler.submit' => ' ', ]; ## DEBUG THE HEADERS debug ( $req->content ); ## MAKE THE ACTUAL REQUEST $res = $ua->request($req); #$loc = $res->headers->header('location'); debug ( $res->content ) if $VERBOSE; ################ DEBUG FUCTION ####################################### +############################################ sub debug { if ($DEBUG) { my $spacer = '|' x 80; print "\n\n$spacer DEBUG $spacer\n"; print @_; print "\n$spacer <END>DEBUG $spacer\n"; } }

Replies are listed 'Best First'.
Re: HTTP::Request Problem
by sgifford (Prior) on Oct 10, 2003 at 17:07 UTC

    The problem is most likely that you're forgetting to set some parameter or cookie, or are setting it to something the Web server doesn't expect. Figuring out exactly what is likely to be time-consuming and tedious; essentially you're doing some minor reverse-engineering on their Web site, and that's always time-consuming.

    One trick that might help is using a packet sniffer to see exactly what a working Web browser is sending and receiving, then looking to see how what you're sending differs from that. Maybe change your script to just send the same as the Web browser does, and when it works start changing one part at a time until you see what breaks.

    Good luck!