Okay, so where's the code?
You just had to follow the link provided by davido to find an example. It took me 5 minutes to change it, despite I never used Mojo before:
use 5.010; use strict; use warnings; use Mojo::UserAgent; use Mojo::IOLoop; use Mojo::URL; # FIFO queue my @names = qw(zezima fred bill john jack); # User agent following up to 5 redirects my $ua = Mojo::UserAgent->new( inactivity_timeout => 1 ); sub url_for_name { my $name = shift; return "http://rscript.org/lookup.php?type=track&time=62899200&use +r=$name&skill=all"; } # Crawler my $crawl; $crawl = sub { my $id = shift; return unless my $name = shift @names; say "Looking for $name"; # Fetch non-blocking just by adding a callback $ua->get( url_for_name($name) => sub { my ( $ua, $tx ) = @_; my $body = $tx->res->body; if ( $body =~ m/gain:Overall:\d+:(\d+)/i ) { say "$name $1"; } elsif ( $body =~ m/(ERROR)/i ) { say "$name doesn't exist"; } else { say "$name 0"; } # Next $crawl->($id); } ); }; # Start a bunch of parallel crawlers sharing the same user agent $crawl->($_) for 1 .. 4; # Start reactor Mojo::IOLoop->start;
And if you do, betcha it takes you 10 times longer to write; requires 10 times... less efficiently...
And that is just rubbish, especially talking about efficiency. I handled hundreds of simultaneous connections with AnyEvent::HTTP, and it didn't really consume a lot of CPU or memory, with threads it would went to swap.
In reply to Re^8: Consumes memory then crashs
by zwon
in thread Consumes memory then crashs
by allhellno
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |