in reply to Re^3: Stateful Browsing and link extraction with AnyEvent::HTTP
in thread Async DNS with LWP

Hi, I've been trying to get parallel DNS working with AnyEvent::DNS. My first poor attempt is effectively the same as synchronous DNS because the program waits for the reply before moving on the next iteration of the loop: 

#!/usr/bin/perl -w
use strict;
use warnings;
use AnyEvent::DNS;

my ($domain);
my (@domains,@condvars);

my $resolver = AnyEvent::DNS::resolver;

while ($domain = <>;)
{
  # clean off newline
  chomp $domain;
  
  # send dns packets
  $resolver->resolve($domain,"*",my $condvar  = AnyEvent->condvar);

  # receive dns packets
  $condvar->recv;
  print "$domain\n";
}

[download]
My second attempt was slightly better in that it can send off ten different DNS packets but receives a reply from the last request ten times (not what I wanted): 

#!/usr/bin/perl -w
use strict;
use warnings;
use AnyEvent::DNS;

my ($domain);
my (@domains,@condvars);

my $resolver = AnyEvent::DNS::resolver;

while (1)
{
  # send dns packets
  for my $i (1..10)
  {
    $domain = <>;
    # clean off newline
    chomp $domain;
    $resolver->resolve($domain,"*",my $condvar  = AnyEvent->condvar);
    push @condvars, $condvar; 
  }
  
  # receive dns packets
  while (my $condvar = pop @condvars)
  {
    $condvar->recv;
    print "$domain\n";
  }
}

[download]
The problem is that $condvar seems to be working like a reference to an object and so each of the 10 $condvars in the stack end up being whatever the last $condvar was. I've tried typeglobbing but this just causes compilation errors. Does anybody know how to do this right?

Replies are listed 'Best First'.
Re: AnyEvent::DNS is effectively synchronous?
by Corion (Patriarch) on Oct 05, 2010 at 11:57 UTC

    You get ten times the "one domain", because you print $domain 10 times. What value would you $domain to have at which time?

    Looking at the documentation of AnyEvent::DNS, the SYNOPSIS section shows how to retrieve results from a query. I'm not sure what problems you have with that.

    If you want to do a callback-driven approach, take a look at the AnyEvent::DNS::srv subroutine. Again, the usage seems quite straightforward, as the resolved IPs get passed as parameters.

[reply]
[d/l]
[select]
      Following is the most simple usage example from the AnyEvent::HTTP documentation. For me this program finishes without any output. Does anybody have any ideas why this doesn't work? 
      #!/usr/bin/perl -w
use strict;
use warnings;
use AnyEvent::HTTP;

http_get "http://www.google.com/", sub { print $_[1] };

      [download]
[reply]
[d/l]

        I guess you need to enter an AnyEvent "main loop" and wait for all asynchronous things to finish. See the AnyEvent documentation:

        use AnyEvent;
use AnyEvent::HTTP;

my $done = AnyEvent->condvar; # stores whether a condition was flagged

http_get "http://www.google.com/", sub { print $_[1]; $done->send };

$done->recv; # enters "main loop" till $condvar gets ->send

        [download]

        Even better, you could send yourself the data through the condvar.

[reply]
[d/l]
      Thanks, in the meantime I'd realised that myself. Just in case it's useful to anybody else I include my code so far for asynchronous resolution. As far as I can see from the documentation retrieving records from the results should be as simple as traversing an array of arrays and picking out the bits you want. e.g. A or AAAA records. 
      
#!/usr/bin/perl -w
use strict;
use warnings;
use Data::Dumper;
use AnyEvent::DNS;

my ($domain,$START_TIME,$MAX_QUERIES,$MAX_QUEUE,
    $time,$done,$resolved);
my (@domains,@condvars);

$START_TIME = time;
$MAX_QUERIES = 1000;
$MAX_QUEUE   = 10;
$resolved = 0;

# setup resolver
my $resolver = AnyEvent::DNS::resolver;
$resolver->max_outstanding($MAX_QUEUE);
#$resolver->timeout([0,1,2]);

while (1)
{
  # send dns packets
  for my $i (1..$MAX_QUERIES)
  {
    $domain = <>;
    # clean off newline
    chomp $domain;
    $resolver->resolve($domain,"*",my $condvar  = AnyEvent->condvar);
    push @condvars, $condvar; 
  }
  
  # receive dns packets
  while (my $condvar = pop @condvars)
  {
    $resolved++ if ($condvar->recv);
    #warn Dumper [$condvar->recv];
  }
  $done += $MAX_QUERIES;
  $time = time - $START_TIME;
  print "Done $done domains, $resolved resolved in $time seconds.\n";
}

      [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom