in reply to Re: Why is LWP::UserAgent so slow?
in thread Why is LWP::UserAgent so slow?

True but LWP::UserAgent is slower than some other methods probably (as already said) down to its flexibility. Not that I am recommending HTTP::Lite (in fact HTTP::Lite pod says use LWP) but to retrieve a file on our local network with LWP::UserAgent and HTTP::Lite where I know it is UTF-8 encoded (so simplified) I get:

use 5.010; use strict; use warnings; use Benchmark; use Encode; use HTTP::Lite; use LWP::UserAgent; my $url = 'xxx.yyy.zzz/file.dat'; sub http_lite { my $r = HTTP::Lite->new; my $sts = $r->request($url); die "failed" if $sts != 200; my $decoded = Encode::decode('utf8', $r->body, Encode::FB_CROAK); } sub lwp_useragent{ my $ua = LWP::UserAgent->new; my $r = $ua->get($url); die "failed" if !$r->is_success; my $body = $r->decoded_content; } timethese(1000, { 'lite' => sub{http_lite()}, 'lwp' => sub{lwp_useragent()} }); Benchmark: timing 1000 iterations of lite, lwp... lite: 3 wallclock secs ( 1.53 usr + 0.60 sys = 2.13 CPU) @ 46 +9.48/s (n=1000) lwp: 6 wallclock secs ( 4.19 usr + 0.58 sys = 4.77 CPU) @ 20 +9.64/s (n=1000)

As I said, I'm not recommending HTTP::Lite just pointing out the difference in a simple case. It does not do HTTPS and defaults to HTTP 1.0 etc and maybe there is something else I've forgotten in the above example that LWP::UserAgent is doing that the HTTP::Lite sub is not.