in reply to Re^19: Async DNS with LWP
in thread Async DNS with LWP
Hi. This is in fact what I have done. First I made a simple serial version based on LWP. This averaged a domain per second (2.8 years for 90,000,000 domains). Then I used Parallel::ForkManager to throw more processes at the job hoping to get an improvement. By tweaking various factors like the number of processes and the length of the timeout I was only able to process up to an average of 10 domains per second. Then I tried with AnyEvent::HTTP, I was able to average about 100 per second. I'm trying to improve on this.