in reply to Re^2: Download web data
in thread Download web data

The Firefox Live HTTP Headers extension is invaluable for this kind of thing. You can watch the requests and responses go by and then write LWP or WWW::Mechanize code to duplicate the sequence.

There are recording proxies that are supposed to do this automatically but I've never had much successs in getting them to work.