Re^3: a crawler

Could be. Do you have any links to them?

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Comment on Re^3: a crawler

Replies are listed 'Best First'.
Re^4: a crawler by planetscape (Chancellor) on Jun 23, 2009 at 19:22 UTC
Here are a few I've found: WWW::Sitemap Building html site maps Web Site Mapper ... though I'd have sworn there were more here on PM... HTH, planetscape	[reply]
Re^5: a crawler by CountZero (Bishop) on Jun 23, 2009 at 20:25 UTC
Oops, and I was even the first to reply on the Building html site maps post. To my defence, it was four years ago ... CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James	[reply]