I would use a hash to store each visited link and use the exists function before visiting a link. The main issues would be:

• You probably want to ensure that you have absolute links (not relative ones), in the hash.
• Ensure that any links that point to scripts have the query string stripped. Rough guess as to the best way to do that:

  $link =~ s/\?.*$//;
  $visited{ $link } = '';
  [download]

Without knowing how you're getting into the infinite loop, I can only offer one other suggestion: for the %visited hash, you may have a problem if a redirection occurs as you'll resolve to the wrong URL. If that's likely to be an issue, you'll have to use LWP::UserAgent and examine the response code.

Cheers,
Ovid

Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.