There is no way that you should using 100% cpu with 10 threads performing IOBut it's not just IO, LWP::UserAgent loads and parses several Perl modules on demand.
For instance, running the OP script (with a fixed set of URLs) under strace on my machine shows that every process reads...
bytes Compress::Raw::Zlib Compress::Zlib Fcntl File::Glob File::GlobMapper File::Spec File::Spec::Unix HTML::Entities HTML::HeadParser HTML::Parser IO IO::Compress::Adapter::Deflate IO::Compress::Base IO::Compress::Base::Common IO::Compress::Gzip IO::Compress::Gzip::Constants IO::Compress::RawDeflate IO::Compress::Zlib::Extra IO::File IO::Handle IO::Seekable IO::Select IO::Socket IO::Socket::INET IO::Socket::UNIX IO::Uncompress::Adapter::Inflate IO::Uncompress::Base IO::Uncompress::Gunzip IO::Uncompress::RawInflate List::Util LWP::Protocol::http Net::HTTP Net::HTTP::Methods Scalar::Util SelectSaver Socket Symbol URI::_generic URI::http URI::_query URI::_server utf8
In reply to Re^2: Parallel::ForkManager (high cpu and a lot of memory)
by salva
in thread Parallel::ForkManager (high cpu and a lot of memory)
by marto9
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |