in reply to Most memory efficient way to threading

A quick check shows that each instance of an LWP::UserAgent costs around 6.8MB. Times 20 and that accounts for 136MB of your 200MB before you start adding in threads, stacks, and queues.

Are you sure that using 20 threads is actually achieving gains over (say) just 10?

If you are just fetching lots of pages in parallel, then you might consider LWP::Parallel::UserAgent.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re: Most memory efficient way to threading

In Section Seekers of Perl Wisdom