• It's not your job to check the links of someone else's site.
• The link checker doesn't respect the Robot Rules, and will wander blindly into robot-hostile territory.

You wouldn't believe the number of lamers that come check stonehenge.com's site hitting thousands of robot-hostile URLs because the first few versions of this program had a live URL pointing at my site. I finally put a very-early block in my website to keep people from doing it. And it still happens! And I no longer use live URLs in my examples. Some people are just Lacking Clue.

-- Randal L. Schwartz, Perl hacker