in reply to Connection Timeout duing form submissions
Hi, did you check the content of dates.txt?
Maybe there's a problem with the line - e.g. like a whitespace somewhere?
The following alternative approach (cut and paste and modify from
lwpcook) worked fine for me...
Update: It directly accesses the service and does not require to fetch and parse the htm file each time.
That should be slightly faster and reduces the network traffic (didn't bench-marked it).
use LWP::UserAgent; $ua = LWP::UserAgent->new; my $req = HTTP::Request->new(POST => 'http://bub2.met.psu.edu/cgi-wi +n/WXDaily.EXE'); $req->content_type('application/x-www-form-urlencoded'); $req->content('dtg=18961110'); my $res = $ua->request($req); print $res->as_string;
However, I am not sure if leeching approx. 41600 pages is a good idea. Maybe your IP or your user-agent is already on the black-list of their web-admin? My advice would be to contact the person responsible for this service and kindly ask for the raw data. Universities usually share such information for research purpose. Don't know what they do if you plan to use this information in a commercial context, though.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Connection Timeout duing form submissions
by cheech (Beadle) on Jun 20, 2009 at 21:14 UTC | |
by Marshall (Canon) on Jun 20, 2009 at 22:53 UTC |