in reply to Why is LWP::UserAgent so slow?

If I write a bash script which iterates 100 times over both scripts I get following results:

You comparing not just the times of sending request, but also the times of scripts compiling. First script imports LWP and JSON, and second only JSON. Look onto my test results:

$ time (for i in `seq 1 100`; do perl -MJSON -e1; done) real 0m2.635s user 0m1.930s sys 0m0.510s $ time (for i in `seq 1 100`; do perl -MJSON -MLWP::UserAgent -e1; don +e) real 0m7.111s user 0m6.060s sys 0m0.740s

Replies are listed 'Best First'.
Re^2: Why is LWP::UserAgent so slow?
by mje (Curate) on Feb 11, 2010 at 09:04 UTC

    True but LWP::UserAgent is slower than some other methods probably (as already said) down to its flexibility. Not that I am recommending HTTP::Lite (in fact HTTP::Lite pod says use LWP) but to retrieve a file on our local network with LWP::UserAgent and HTTP::Lite where I know it is UTF-8 encoded (so simplified) I get:

    use 5.010; use strict; use warnings; use Benchmark; use Encode; use HTTP::Lite; use LWP::UserAgent; my $url = 'xxx.yyy.zzz/file.dat'; sub http_lite { my $r = HTTP::Lite->new; my $sts = $r->request($url); die "failed" if $sts != 200; my $decoded = Encode::decode('utf8', $r->body, Encode::FB_CROAK); } sub lwp_useragent{ my $ua = LWP::UserAgent->new; my $r = $ua->get($url); die "failed" if !$r->is_success; my $body = $r->decoded_content; } timethese(1000, { 'lite' => sub{http_lite()}, 'lwp' => sub{lwp_useragent()} }); Benchmark: timing 1000 iterations of lite, lwp... lite: 3 wallclock secs ( 1.53 usr + 0.60 sys = 2.13 CPU) @ 46 +9.48/s (n=1000) lwp: 6 wallclock secs ( 4.19 usr + 0.58 sys = 4.77 CPU) @ 20 +9.64/s (n=1000)

    As I said, I'm not recommending HTTP::Lite just pointing out the difference in a simple case. It does not do HTTPS and defaults to HTTP 1.0 etc and maybe there is something else I've forgotten in the above example that LWP::UserAgent is doing that the HTTP::Lite sub is not.

Re^2: Why is LWP::UserAgent so slow?
by smetj (Initiate) on Feb 18, 2010 at 19:37 UTC
    Yep indeed, I didn't take the compile time into account. The larger the modules, the more time it takes obviously.. thanks