Hi mischief,

I don't think this is correct - I am trying to do things this way for that reason specifically.

If I didn't make the URLs unique before passing them to the Net::DNS::Resolver module I would be doing even more duplication (some duplication is unavoidable as you correctly pointed out).

By resolving only unique URLs I am minimizing the amount of DNS resolution that is required. It would be wasteful to resolve the exact same URL multiple times as it would always yield the same result.