Occasionally I think of stupid hacks to accomplish things. Most of the time I remind myself that they are stupid, and they never see the light of day. But just to be silly I will tell you a way that you can get your script to act just like a browser, complete with proper handling of frames, JavaScript, meta tags, cookies and the like.

Go find an http proxy in Perl. (merlyn's columns would be a good place to start.) Modify it to parse the documents passing through to find links in them. (HTML::LinkExtor may help here.) Then set a copy of IE to use that as your proxy server. Use the OLE modules to drive IE around the web.

Voila! :-)

If you just need an answer to a question, this might be acceptable. But I (obviously) wouldn't put this into production. (Says the man who once had a temporary hack which used IE to produce PDFs left in production for half a year...)