Herkum has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to get LWP::UserAgent to work on my system with a hostname. No matter what hostname I use I ALWAYS get the same error.

#!/usr/bin/perl -w
use strict;

use Data::Dump qw(dump);
use LWP::UserAgent;

my $ua = LWP::UserAgent->new;
   $ua->timeout(10);

my $req = HTTP::Request->new(GET => 'http://my.yahoo.com');
warn "Dump request: " . dump( $req );

my $res = $ua->request($req);
warn "Dump Response: " . dump( $res );

[download]

I get this error "500 Can't connect to my.yahoo.com:80 (Bad hostname 'my.yahoo.com')". It does not matter what hostname I use I always get the same result. If I use a specific IP then it will do a valid request but I want to make this work with DNS names.

The bizarre thing is that all the other utilities and my web browser don't seem to have a problem just LWP::UserAgent. I guess that I don't know where it is getting its environment information to determine the hostname.

I am using perl 5.8.8 on Ubuntu, does anyone have any ideas on what I can check?

Update: I decided to upgrade the libraries and noticed that it skipped some testings during the install. It skipped local/http.t and 3 others all with the same message (misconfigured system). If this is a systems issue, I know i am screwed because I have not been a SysAdmin for years.

Replies are listed 'Best First'.
Re: LWP::UserAgent always complains about Hostname
by jethro (Monsignor) on Jul 19, 2008 at 00:14 UTC
    You might single-step through that http.t with the perl debugger to find out how it knows that yours is a misconfigured system

    IMHO easier would be to install perl 10.0 in /usr/local and try your script with that. If you do that, you find out whether ubuntu or perl5.8.8 is the culprit. While installing LWP::UserAgent you also find out on which non-core modules UserAgent is based and maybe some of them sound like they provide name services. Any likely candidates can then be reinstalled in your perl5.8.8 installation.

    Just be careful with CPAN and both perl version. While different perl version have no problem coexisting on one machine it seems to me that cpan is using only one .cpan directory and can't cope with supplying more than one perl version. It might be better to create a second account just for using cpan with perl10.0 and leaving the old account strictly cpan with perl5.8.8

    You might also use tcpdump to monitor whether there is any communication over the network when your script is called. Check the logfiles. Check /etc/nsswitch.conf and /etc/resolv.conf for sane values.

[reply]

      install perl 10.0

      find out on which non-core modules UserAgent is based

      These two things I can look into. They should not be that hard, at least compared with moving to a new system.

[reply]
        Hi, I'm sure you must have already sorted out this issue by this time. But I just wanted to share my problem too here, so others can have this information. Also, please share your solution on this issue. I had the similar issue with LWP.UserAgent complaining "bad hostname" for whatever the host we give irrespective of the proxy url setup on the agent. All the hosts works just fine when we access them in the browser. We get response to nslookup and ping for all the hosts. Strange thing is that (my script) same script worked just fine when I tried with v5.8.8 and tomcat apache on solaris. But the same script fails when I use ActivePerl on windows xp. I tried to google for the solution and possible causes of this issue, but no luck :( My solution: I tried the same script in one of my friend's desktop where he was using apache instead of tomcat apache with the same ActivePerl. It just worked fine. So, I too ended up in installing and using apache instead of tomcat apace which worked well for me too. If somebody has more information on this issue and a resolution for this, please share! Thanks A PERL newbie!
[reply]
Re: LWP::UserAgent always complains about Hostname
by Khen1950fx (Canon) on Jul 18, 2008 at 23:35 UTC
    I downloaded the source and installed it. "local/http.t" passed. Then I ran runprove on local/http.t, and I got "Can't talk to ourself(misconfigured system)". Strange because your script worked for me without problems. I don't think that it's the system, but something in the way the test is set up:).
[reply]
Re: LWP::UserAgent always complains about Hostname
by alexm (Chaplain) on Jul 19, 2008 at 13:35 UTC
    Maybe there's something wrong with your %Net::Config::NetConfig or you have defined a proxy that doesn't work properly? 
    $ perl -MNet::Config=%NetConfig -MData::Dumper -e 'print Dumper(\%NetC
+onfig)'
$ env | grep proxy

    [download]
[reply]
[d/l]
[select]

      I ran your script and got this back.

      $VAR1 = {
          'pop3_hosts' => [],
          'ftp_firewall' => undef,
          'ph_hosts' => [],
          'inet_domain' => undef,
          'time_hosts' => [],
          'daytime_hosts' => [],
          'smtp_hosts' => [],
          'test_exist' => 1,
          'test_hosts' => 1,
          'nntp_hosts' => [],
          'ftp_ext_passive' => 1,
          'snpp_hosts' => [],
          'ftp_testhost' => undef,
          'ftp_int_passive' => 0
        };

      [download]

      Most of those options, at least to me, seem non-critical for DNS resolution. As for the proxy variables, there no variables set and I am not using a proxy server either.

[reply]
[d/l]

        Okay, then... Can you try this?

        $ strace -o 698729.out -econnect,sendto,recvfrom perl 698729.pl

        [download]

        The 698729.out file should start with something similar to this:

        connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1
+ ENOENT (No such file or directory)
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1
+ ENOENT (No such file or directory)
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(
+"192.168.1.1")}, 28) = 0
sendto(3, "\321\263\1\0\0\1\0\0\0\0\0\0\2my\5yahoo\3com\0\0\1\0\1"...,
+ 30, MSG_NOSIGNAL, NULL, 0) = 30
recvfrom(3, "\321\263\201\200\0\1\0\2\0\0\0\0\2my\5yahoo\3com\0\0\1"..
+., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_add
+r("192.168.1.1")}, [16]) = 80
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr(
+"216.252.123.25")}, 16) = -1 EINPROGRESS (Operation now in progress)
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr(
+"216.252.123.25")}, 16) = 0
...

        [download]

        Note that 192.168.1.1 is my DNS nameserver and I don't have nscd running, so the fist ENOENT errors can be ignored. You can see that the DNS query got through since 216.252.123.25 is the IP address for my.yahoo.com and there's a connection to port 80 going on.

        Hope that helps.

        Update: renamed lwp-useragent.pl to OP's node ID, so it's clear where does the code come from.

[reply]
[d/l]
[select]
Re: LWP::UserAgent fails with really long lines in /etc/hosts
by dec (Beadle) on Apr 14, 2009 at 00:06 UTC
    Similar problem with my own LWP::UserAgent script and the hearse package on Ubuntu. In my case the solution lay in my /etc/hosts file. I had a lot of troublesome sites aliased to 127.0.0.1, each on a separate line. At some point (probably hitting the "hosts" tab of the network manager applet) all these separate lines got rolled up into one really long line. Replacing the long line with 127.0.0.1 localhost fixed the problem. Looks like the ns library chokes with really long lines in /etc/hosts, which causes a knock-on failure with Perl's resolver.
[reply]

Back to Seekers of Perl Wisdom