Hi Corion, thanks for the tip
I am currently using Parallel::ForkManager to make the process faster but will also take a look at AnyEvent::HTTP
The HEAD option, however, does not seem reduce to the time of response so I am looking for new ideas
Guy
Comment on Re^2: Fast fetching of HTML response code
You likely can't reduce the time per URL, but you can reduce the total time by making more requests in parallel. Or you can try to find out whether you are bandwidth-limited, or CPU-limited or whether the remote side is slow to respond. My guess is that the remote side is slow to respond.