Is the webserver running under your control? Some ISPs put a time limit on running cgi-scripts. If your script takes too long on a large amount of content, it may be killed off by the server (SIGTERM....). HTH, although it is not really a debugging hint...
LWP::UserAgent does have a timeout, but I have increased it to 20 minutes. I have also tcpdumped the interface and it is making connections rapidly in succession with no errors. The tcp sessions are also shutting down cleanly.