in reply to Windows: AnyEvent -> HTTP -> DNS - > Blocking for minutes

means I can't redefine the _defaults function, as it gets re-defined each time AnyEvent::DNS calls it.

Can you explain that? From what I see in AnyEvent::DNS, it is only using Net::DNS::Resolve here:

require Net::DNS::Resolver; my $r = Net::DNS::Resolver->new; $r->nameservers or die; for my $s ($r->nameservers) { if (my $ipn = AnyEvent::Socket::parse_address ($s)) { push @{ $self->{server} }, $ipn; } } $self->{search} = [$r->searchlist];
and $r goes out of scope after that. There should not be any problem monkey-patching functions inside of Net::DNS::Resolver, or even AnyEvent::DNS. But you don't need to monkey-patch it if you know your resolvers, because:
AnyEvent::DNS::resolver This function creates and returns a resolver that is ready to use and should mimic the default resolver for your system as good as possible. It is used by AnyEvent itself as well. It only ever creates one resolver and returns this one on subsequent calls - see $AnyEvent::DNS::RESOLVER, below, for details. Unless you have special needs, prefer this function over creating your own resolver object. The resolver is created with the following parameters: untaint enabled max_outstanding $ENV{PERL_ANYEVENT_MAX_OUTSTANDING_DNS} (default 10) os_config will be used for OS-specific configuration, unless $ENV{PERL_ANYEVENT_RESOLV_CONF} is specified, in which case that file gets parsed. $AnyEvent::DNS::RESOLVER This variable stores the default resolver returned by AnyEvent::DNS::resolver, or undef when the default resolver hasn't been instantiated yet. One can provide a custom resolver (e.g. one with caching functionality) by storing it in this variable, causing all subsequent resolves done via AnyEvent::DNS::resolver to be done via the custom one.
So it appears you have lots of options to customize this.

Meanwhile, AnyEvent::DNS doesn't use Net::DNS::Resolver for DNS lookups, it only uses it to dig the nameserver list and search path out of the Windows registry. Are you sure that your program is hanging in the place that you think it's hanging?

Replies are listed 'Best First'.
Re^2: Windows: AnyEvent -> HTTP -> DNS - > Blocking for minutes
by sectokia (Friar) on Oct 19, 2022 at 09:28 UTC

    Thanks for response, sorry I took so long to reply I have been ill.

    Here is simple test that causes the issue on windows (disable all network adaptors on windows before running, so there are no DNS servers):

    use AnyEvent; use AnyEvent::HTTP; my $cv = AnyEvent->condvar; my $timer = AnyEvent->timer(after=>1,interval=>1,cb=>sub{ print STDOUT + "."; flush STDOUT; }); my $sig = AnyEvent->signal (signal => "INT", cb => sub { $cv->send; }) +; http_get "http://www.google.com/", sub { print "Get completed (".$_[1]->{Status}.")"; $cv->send; }; $cv->recv;

    I have figured out the call chain, but I am unable to determine what to do about it. It seems like Resolver is trying to send a udp packet to a blank IP address?

    Anyevent::DNS is doing this:

    my $r = Net::DNS::Resolver->new;

    Net::DNS::Resolver::MSWin32 in its _init is calling:

    $defaults->nameservers(@nameservers);

    nameservers() is in Net::DNS::Resolver::Base which calls send() and _send_udp()

    _send_udp is then calling can_read from IO::Select

    This results in a blocking call at this line with the timeout being several minutes:

    defined($r) && (select($r,undef,undef,$timeout) > 0)

    I cannot figure out what resolver is trying to do here or who it is even trying to send to. Any help would be great! Thanks

      So, looking at Net::DNS::Resolver::Base::nameservers(), I see one code path that tries to run a name lookup:
      foreach my $ns ( grep {defined} @_ ) { if ( _ipv4($ns) || _ipv6($ns) ) { push @ip, $ns; } else { my $defres = ref($self)->new( debug => $self->{debug} ); $defres->{persistent} = $self->{persistent}; my $names = {}; my $packet = $defres->send( $ns, 'A' );

      That would happen if the nameserver list passed to  ->nameservers(@list) contained a string that wasn't an IP address. It seems really odd to me that name servers would ever be specified as names that needed resolved before they could be used to resolve names... maybe you should debug that first to see what the input is?

      Try this: (example of monkey-patching)

      use Net::DNS::Resolver::Base; my $orig= \&Net::DNS::Resolver::Base::nameservers; local *Net::DNS::Resolver::Base::nameservers= sub { my ($self, @servers)= @_; print "Current nameservers value: " .join(", ", @{$self->{nameservers}//[]}) ."\n"; print "Assigning nameservers: ".join(", ", @servers)."\n" if @servers > 1; $orig->(@_); }; # and then the rest of your program

      I don't have a windows system to test on at the moment (well, not one where I can disable the network adapters and still use it) but that will at least show you what is getting assigned there. On my system, it reveals that the default resovlers are "127.0.0.1" and "::1" before the first call to ->nameservers, so yes that would hang if there was not a nameserver listening on localhost and something tried to assign "foo.example.com" as a nameserver.

      Meanwhile, I realized that you can override the default timeouts like this:

      use Net::DNS::Resolver; Net::DNS::Resolver->tcp_timeout(1); Net::DNS::Resolver->udp_timeout(1); # remainder of your program
      That effect comes from the AUTOLOAD method on ::Base which generates accessors on demand:
      sub AUTOLOAD { ## Default method my ($self) = @_; my $name = $AUTOLOAD; $name =~ s/.*://; croak qq[unknown method "$name"] unless $public_attr{$name}; no strict 'refs'; ## no critic ProhibitNoStrict *{$AUTOLOAD} = sub { my $self = shift; $self = $self->_defaults unless ref($self); $self->{$name} = shift || 0 if scalar @_; return $self->{$name}; }; goto &{$AUTOLOAD}; }
      so any time you call an accessor on the package itself, it pulls up the _default resolver and sets the attribute on it instead. Those attributes get used as the defaults for any new resolvers created.
        It seems really odd to me that name servers would ever be specified as names that needed resolved before they could be used to resolve names...

        This is used in the testing/using of authoritative and/or delegated nameservers which are usually specified by name rather than by address (so that they may be rotated, migrated, etc. by the hoster without affecting the delegation). So long as you have bootstrapped correctly by having an initial resolver to do the lookups of the nameserver names it is a handy feature.


        🦛

        Yes for me I get attempts to 127.0.0.1 and ::1 as well

        Setting tcp_timeout and udp_timeout changed nothing for me. Having a look in _udp_send the timeout used for can_read is made from the 'retrans' and 'retry' settings. Based on the default values (5 and 4) with the DNS timeout of 2.5s doubling, this results in a timeouts of: 2.5,2.5,5,5,10,10,20,20 for both servers (total 150 seconds).

        This reduced the issue to 2.5 seconds:

        Net::DNS::Resolver->tcp_timeout(1); Net::DNS::Resolver->udp_timeout(1);

        But then I noticed that the local host name servers are in the defaults as well! So this 'fixes' it by setting the default nameservers to nothing:

        Net::DNS::Resolver->nameserver4([]); Net::DNS::Resolver->nameserver6([]);