There's a couple of different places it could time out.

• The browser. IE waits about 30s econds for a response before giving up
• The OS socket itself may have a timout.
• The webserver may have a time out.

But, the bottom line is, as long as your CGI outputs SOMETHING then these timeouts are reset...that's why your rewritten code works. It produces something sooner, thus eliminating the timeout.

The webserver knows your script is done, when it stop running :) not by any sentinal output.

If you took your original script and changed it to

$|++;  # turn off output buffering.
for each page-I-want-to-get{
  print "getting page"
  get it using LWP:Simple or skip if error
  print "done"
  munge the HTML
  write it out to a file
}
[download]

This should prevent the timeout from occuring.

HTH

/\/\averick
perl -l -e "eval pack('h*','072796e6470272f2c5f2c5166756279636b672');"