in reply to Re: Question: Fast way to validate 600K websites
in thread Question: Fast way to validate 600K websites

Hi, huys:

Thank you all for the helpful suggestions:-) I am actually trying to check if 600K listed domain names are reachable. many of them are just garbages like 0.00, hotmailll.com. so I need to discard them(like 000.0.com) or correct them(i.e. from 'hotmaillll.com' to 'hotmail.com'). Right now I have not yet consider sites which disable 'HEAD' method. at this stage, I will just filter out those 'NOT valid' sites into a list and then do more search on that smaller list. :)Most of the information I got so far from this thread is very helpful, thanks again: )

lihao

  • Comment on Re^2: Question: Fast way to validate 600K websites

Replies are listed 'Best First'.
Re^3: Question: Fast way to validate 600K websites
by leocharre (Priest) on May 12, 2008 at 20:38 UTC
    If you want to know if the uri is actually reachable, would simple posix 'ping' help you?