Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Windows: AnyEvent -> HTTP -> DNS - > Blocking for minutes

by sectokia (Pilgrim)
on Oct 12, 2022 at 03:08 UTC ( [id://11147363]=perlquestion: print w/replies, xml ) Need Help??

sectokia has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

On windows, perl will block completely for many minutes if there is no connection to a DNS server when you are using AnyEvent::HTTP. The issue seems to be AnyEvent::HTTP uses AnyEvent::DNS which on windows doesn't have a proper async way to make DNS calls and instead falls back to Net::DNS::Resolver (which is blocking) and AnyEvent::DNS doesn't otherwise set timeouts, so the default timeouts occur (which are several minutes).

This means if there is no connection to the DNS server, then all events under AnyEvent can't run until the timeout elapses after a few minutes.

I thought I could cludge around this somewhat by lowering the timeouts, but the dynamic way in which Net::DNS::Resolve is called means I can't redefine the _defaults function, as it gets re-defined each time AnyEvent::DNS calls it.

Anyone got any ideas to work around this? Thanks

  • Comment on Windows: AnyEvent -> HTTP -> DNS - > Blocking for minutes

Replies are listed 'Best First'.
Re: Windows: AnyEvent -> HTTP -> DNS - > Blocking for minutes
by NERDVANA (Deacon) on Oct 13, 2022 at 03:06 UTC
    means I can't redefine the _defaults function, as it gets re-defined each time AnyEvent::DNS calls it.

    Can you explain that? From what I see in AnyEvent::DNS, it is only using Net::DNS::Resolve here:

    require Net::DNS::Resolver; my $r = Net::DNS::Resolver->new; $r->nameservers or die; for my $s ($r->nameservers) { if (my $ipn = AnyEvent::Socket::parse_address ($s)) { push @{ $self->{server} }, $ipn; } } $self->{search} = [$r->searchlist];
    and $r goes out of scope after that. There should not be any problem monkey-patching functions inside of Net::DNS::Resolver, or even AnyEvent::DNS. But you don't need to monkey-patch it if you know your resolvers, because:
    AnyEvent::DNS::resolver This function creates and returns a resolver that is ready to use and should mimic the default resolver for your system as good as possible. It is used by AnyEvent itself as well. It only ever creates one resolver and returns this one on subsequent calls - see $AnyEvent::DNS::RESOLVER, below, for details. Unless you have special needs, prefer this function over creating your own resolver object. The resolver is created with the following parameters: untaint enabled max_outstanding $ENV{PERL_ANYEVENT_MAX_OUTSTANDING_DNS} (default 10) os_config will be used for OS-specific configuration, unless $ENV{PERL_ANYEVENT_RESOLV_CONF} is specified, in which case that file gets parsed. $AnyEvent::DNS::RESOLVER This variable stores the default resolver returned by AnyEvent::DNS::resolver, or undef when the default resolver hasn't been instantiated yet. One can provide a custom resolver (e.g. one with caching functionality) by storing it in this variable, causing all subsequent resolves done via AnyEvent::DNS::resolver to be done via the custom one.
    So it appears you have lots of options to customize this.

    Meanwhile, AnyEvent::DNS doesn't use Net::DNS::Resolver for DNS lookups, it only uses it to dig the nameserver list and search path out of the Windows registry. Are you sure that your program is hanging in the place that you think it's hanging?

      Thanks for response, sorry I took so long to reply I have been ill.

      Here is simple test that causes the issue on windows (disable all network adaptors on windows before running, so there are no DNS servers):

      use AnyEvent; use AnyEvent::HTTP; my $cv = AnyEvent->condvar; my $timer = AnyEvent->timer(after=>1,interval=>1,cb=>sub{ print STDOUT + "."; flush STDOUT; }); my $sig = AnyEvent->signal (signal => "INT", cb => sub { $cv->send; }) +; http_get "http://www.google.com/", sub { print "Get completed (".$_[1]->{Status}.")"; $cv->send; }; $cv->recv;

      I have figured out the call chain, but I am unable to determine what to do about it. It seems like Resolver is trying to send a udp packet to a blank IP address?

      Anyevent::DNS is doing this:

      my $r = Net::DNS::Resolver->new;

      Net::DNS::Resolver::MSWin32 in its _init is calling:

      $defaults->nameservers(@nameservers);

      nameservers() is in Net::DNS::Resolver::Base which calls send() and _send_udp()

      _send_udp is then calling can_read from IO::Select

      This results in a blocking call at this line with the timeout being several minutes:

      defined($r) && (select($r,undef,undef,$timeout) > 0)

      I cannot figure out what resolver is trying to do here or who it is even trying to send to. Any help would be great! Thanks

        So, looking at Net::DNS::Resolver::Base::nameservers(), I see one code path that tries to run a name lookup:
        foreach my $ns ( grep {defined} @_ ) { if ( _ipv4($ns) || _ipv6($ns) ) { push @ip, $ns; } else { my $defres = ref($self)->new( debug => $self->{debug} ); $defres->{persistent} = $self->{persistent}; my $names = {}; my $packet = $defres->send( $ns, 'A' );

        That would happen if the nameserver list passed to  ->nameservers(@list) contained a string that wasn't an IP address. It seems really odd to me that name servers would ever be specified as names that needed resolved before they could be used to resolve names... maybe you should debug that first to see what the input is?

        Try this: (example of monkey-patching)

        use Net::DNS::Resolver::Base; my $orig= \&Net::DNS::Resolver::Base::nameservers; local *Net::DNS::Resolver::Base::nameservers= sub { my ($self, @servers)= @_; print "Current nameservers value: " .join(", ", @{$self->{nameservers}//[]}) ."\n"; print "Assigning nameservers: ".join(", ", @servers)."\n" if @servers > 1; $orig->(@_); }; # and then the rest of your program

        I don't have a windows system to test on at the moment (well, not one where I can disable the network adapters and still use it) but that will at least show you what is getting assigned there. On my system, it reveals that the default resovlers are "127.0.0.1" and "::1" before the first call to ->nameservers, so yes that would hang if there was not a nameserver listening on localhost and something tried to assign "foo.example.com" as a nameserver.

        Meanwhile, I realized that you can override the default timeouts like this:

        use Net::DNS::Resolver; Net::DNS::Resolver->tcp_timeout(1); Net::DNS::Resolver->udp_timeout(1); # remainder of your program
        That effect comes from the AUTOLOAD method on ::Base which generates accessors on demand:
        sub AUTOLOAD { ## Default method my ($self) = @_; my $name = $AUTOLOAD; $name =~ s/.*://; croak qq[unknown method "$name"] unless $public_attr{$name}; no strict 'refs'; ## no critic ProhibitNoStrict *{$AUTOLOAD} = sub { my $self = shift; $self = $self->_defaults unless ref($self); $self->{$name} = shift || 0 if scalar @_; return $self->{$name}; }; goto &{$AUTOLOAD}; }
        so any time you call an accessor on the package itself, it pulls up the _default resolver and sets the attribute on it instead. Those attributes get used as the defaults for any new resolvers created.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11147363]
Approved by Athanasius
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (5)
As of 2024-04-24 22:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found