for each page-I-want-to-get{
  check it's up using LWP:Simple's head(); or skip if error
}
output a new "index.cgi" page with links to all the files which are 
*about* to be created from the pages which passed the test 
above

then go and actually get the pages' html , munge it and write the
files