add_content is called in a loop by the collect method of LWP::Protocol. I was able to reduce the download time for a 12MB file from 36 seconds to 5 seconds by changing the collect method to use an array for the buffer and then stick everything together at the end, instead of calling add_content repeatedly:$self->{_content} .= $$chunkref;
Anybody know any reason that would be a problem?my @total_content; if (!defined($arg) || !$response->is_success) { # scalar while ($content = &$collector, length $$content) { if ($parser) { $parser->parse($$content) or undef($parser); } LWP::Debug::debug("read " . length($$content) . " bytes"); push(@total_content, $$content); $content_size += length($$content); $ua->progress(($length ? ($content_size / $length) : "tick"), +$response); if (defined($max_size) && $content_size > $max_size) { LWP::Debug::debug("Aborting because size limit exceeded"); $response->push_header("Client-Aborted", "max_size"); last; } } $response->add_content(join('', @total_content));
In reply to Re^2: LWP slow downloads on windows (50 times faster?)
by robobunny
in thread LWP slow downloads on windows
by robobunny
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |