Re: Using Threading to speed up DNS Resolution

This is more work than you probably realize. Most name resolver libraries are single-threaded no matter what you do -- that is, regardless of whether you spawn off a zillion threads there'll only be one outstanding name lookup request for your process. If this is the case for you (and it may well be. Check before you do anything) then you'll need to find or write your own name resolution library. It's not horribly tough, but it does mean learning how to do DNS' wire protocol.

Running a local name server and talking to it rather than some remote server will probably speed things up quite a bit as well.

Comment on Re: Using Threading to speed up DNS Resolution

Replies are listed 'Best First'.
Re^2: Using Threading to speed up DNS Resolution by jfroebe (Parson) on Oct 22, 2004 at 16:41 UTC
Actually most gethostby_r() implementations can be used with threading with no significant issues. Perhaps you are thinking of the non reentrant gethostby()? No one has seen what you have seen, and until that happens, we're all going to think that you're nuts. - Jack O'Neil, Stargate SG-1	[reply]
Re^3: Using Threading to speed up DNS Resolution by Elian (Parson) on Oct 22, 2004 at 18:31 UTC
No, you misunderstand. It's not that the interface isn't threadsafe. It's that the implementation is single-threaded. That is, no matter how many threads are making name lookup requests, there'll only be one actually in-flight at any one time. Having 50 threads making simultaneous name lookups will get you 1 thread doing a name lookup and 49 threads waiting their turn to issue the name lookup.	[reply]