web dead link check spider 2008

cutlass2006 has asked for the wisdom of the Perl Monks concerning the following question:

everytime I launch a new site, I ask myself the question 'how do I find dead links' and end up writing a tiny perl spider script to do the job.

I seek if anyone else has a 'better' perl script for checking web links and generally performing a web lint type analysis.

update: thanks for the knowledge refresh ... WWW::CheckSite::Validator was especially helpful

tia, JimF

Comment on web dead link check spider 2008

Replies are listed 'Best First'.
Re: web dead link check spider 2008 by lamp (Chaplain) on Sep 19, 2008 at 10:27 UTC
WWW::CheckSite::Validator module can help you better for validating the website.	[reply]
Re: web dead link check spider 2008 by moritz (Cardinal) on Sep 19, 2008 at 10:58 UTC
On Debian Etch: `linkchecker - check websites and HTML documents for broken links w3c-linkchecker - W3C Link Checker htcheck - Utility for checking web site for dead/external links klinkstatus - web link validity checker for KDE linklint - A fast link checker and web site maintenance tool webcheck - website link and structure checker` [download]	[reply] [d/l]
Re: web dead link check spider 2008 by stark (Pilgrim) on Sep 19, 2008 at 17:03 UTC
You could also use Test::WWW::Mechanize It offers the functions page_links_ok() and html_lint_ok() among many others...	[reply]