Hello Monks,

Today i would like to give you some first impressions of WWW::Curl::Lite.
Here are some examples to wet your appetite.

Oneliner:
perl -MWWW::Curl::Lite -e 'print WWW::Curl::Lite->get("http://files.oo
+ok.de")'

[download]

The advanced interface:
 #!/usr/bin/perl -w
use strict;
use WWW::Curl::Lite;
use WWW::Curl::Lite::Request;

my $request = new WWW::Curl::Lite::Request({ url => 'http://oook.de' }
+);
$request->useragent('curltest/0.1');
$request->postfields('test=1&curl=cool');

my $curl = new WWW::Curl::Lite;
my $response = $curl->request($request);
print $response->content;

[download]

The parallel interface:
#!/usr/bin/perl -w
use strict;
use WWW::Curl::Lite;
use WWW::Curl::Lite::Request;

my $request1 = new WWW::Curl::Lite::Request;
$request1->url('http://oook.de');

my $request2 = new WWW::Curl::Lite::Request;
$request2->url('http://files.oook.de');

my $curl = new WWW::Curl::Lite;
$curl->register($request1);
$curl->register($request2);
my $responses = $curl->request;
foreach my $response (@$responses) {
    print $response->content;
}

[download]

Some of you may already know WWW::Curl, and how horrible it's API is.
Here is a small example that shows it.

#!/usr/bin/perl -w
use strict;
use WWW::Curl::Easy;

my $url = 'http://oook.de';

sub header_callback {
   my $chunk = shift;
   print($chunk);
   return length($chunk);
}
                                                                      
+     
sub body_callback {
   my ( $chunk, $handle ) = @_;
   print($chunk);
   return length($chunk);
}

my $curl = new WWW::Curl::Easy;
$curl->setopt( CURLOPT_URL,            $url );
$curl->setopt( CURLOPT_HEADERFUNCTION, \&header_callback );
$curl->setopt( CURLOPT_WRITEFUNCTION,  \&body_callback );
$curl->perform;

[download]
Yes, thats very similar to libcurls C API, but pure horror for most Perl hackers. :)

In the future WWW::Curl::Lite should become a better alternative to LWP.
Currently it's biggest feature is its performance, benchmarks showed that its around 30 times faster than LWP and LWP::Parallel.

I have not yet benchmarked it against HTTP::GHTTP and HTTP::MHTTP, but WWW::Curl::Lite can handle parallel downloads, so chances are very good that it will win.

WWW::Curl::Lite doesn't use the official WWW::Curl 2.0 from CPAN, instead it uses my unofficial WWW::Curl 3.0, which adds many new features, the most important one is multi support.
Next to that many things were cleaned or rewritten from scratch.

WWW::Curl::Lite and WWW::Curl 3.0 are not yet on CPAN, because of some problems with the upstream maintainer, but i hope that will change very soon.
UPDATE: Problems are solved, we are working on a release.

You would help very much by giving some productive feedback, like whats missing or what sucks.

Patches, documentation or some evil examples would be nice too!

Currently you can grab actual versions from here:
WWW::Curl::Lite
WWW::Curl

Replies are listed 'Best First'.
Re: First impressions of WWW::Curl::Lite
by Corion (Patriarch) on Feb 16, 2004 at 07:04 UTC

    This is very interesting, but I think it will never replace LWP, as LWP is pure Perl and thus requires no C compiler for installation, which is very convenient (and more convenient than shelling out to wget).

    What I would like would be a drop-in compatibility layer so that I can replace my use of LWP::Simple, or even better the use of LWP::UserAgent by WWW::Curl::* .

[reply]
Re: First impressions of WWW::Curl::Lite
by Abigail-II (Bishop) on Feb 16, 2004 at 08:37 UTC
    Currently it's biggest feature is its performance, benchmarks showed that its around 30 times faster than LWP and LWP::Parallel.

    Yes, and where are those benchmarks?

    Abigail

[reply]
      Here is a quick Benchmark:
      #!/usr/bin/perl -w
use strict;
use Benchmark qw(:all);
use WWW::Curl::Lite::Request;
use WWW::Curl::Lite;
use LWP::UserAgent;
use LWP::Parallel::UserAgent;
use HTTP::Request;

my $request = new WWW::Curl::Lite::Request( { url => 'http://127.0.0.1
+' } );
my $curl    = new WWW::Curl::Lite;

my $ua = new LWP::UserAgent;

my $pua     = new LWP::Parallel::UserAgent;
my $httpreq = new HTTP::Request( GET => 'http://127.0.0.1' );

timethese(
    100,
    {
        'curl' => sub {
            for ( 1 .. 10 ) { $curl->register($request) }
            $curl->request;
        },
        'lwp' => sub {
            for ( 1 .. 10 ) { $ua->get('http://127.0.0.1') }
        },
        'lwp-parallel' => sub {
            $pua->initialize;
            $pua->nonblock(1);
            for ( 1 .. 10 ) { $pua->register($httpreq) }
            $pua->wait;
          }
    }
);

      [download]
      sri@odyssey:~$ perl benchmark2.pl
Benchmark: timing 100 iterations of curl, lwp, lwp-parallel...
      curl: 25 wallclock secs ( 1.07 usr +  0.18 sys =  1.25 CPU) @ 80
+.00/s (n=100)
       lwp: 30 wallclock secs ( 7.12 usr +  0.51 sys =  7.63 CPU) @ 13
+.11/s (n=100)
lwp-parallel: 34 wallclock secs (11.07 usr +  0.56 sys = 11.63 CPU) @ 
+ 8.60/s (n=100)

      [download]
      Most suprising is how bad LWP::Parallel performed.

      Yes, it wasn't 30 times faster in this benchmark, the original one was performed on a small crawler cluster with 10000 real world url's, maybe i can reformat these results and post them too.
[reply]
[d/l]
[select]

        I'm not a benchmark guru, but I'd also like to see HTTP::MHTTP, HTTP::GHTTP and HTTP::Lite included in the benchmark too.

        In my experience, MHTTP is the fastest, GHTTP is pretty close, and Lite is slower, but still faster than LWP. Like LWP, Lite is also pure Perl, and doesn't require a backend c library.


        --
        ajt
[reply]
        That seems to unfairly penalize LWP by using a shortcut method that constructs a request on every iteration while you pre-construct it for the other methods.

        Makeshifts last the longest.

[reply]

Back to Meditations