LWP running as cgi

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Greets Monks,
I'm fairly sure this isn't a perl problem per-se, but more of a configuration issue, so please don't flame me, as I'm sure one of you might have seen this before, or can tell me how to track it down
I'm running a little LWP script as cgi. It works when run from the command line, but dies when run as cgi with:
LWP result: {500 Can't connect to www.yahoo.com:80 (Bad hostname 'www.yahoo.com')}

#!/usr/bin/perl
use LWP::Debug qw(level); level('+');
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->timeout(10);
my $resp = $ua->get("http://www.yahoo.com/");
my $resultUsingLWP;
if($resp->is_success) {
$resultUsingLWP = $resp->content;
} else {
$resultUsingLWP = $resp->status_line;
}
print "Content-type: text/html; charset=utf-8\n\n";
print "LWP result: {$resultUsingLWP}";
print "\n";
[download]

I'm currently thinking it could be the paths which are:
root (command line): /usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
apache (cgi): /usr/bin:/bin
however, as all other cgi rograms work fine, is there something in LWP that needs access to one of these directories?

Comment on LWP running as cgi Download Code

Replies are listed 'Best First'.
Re: LWP running as cgi by merlyn (Sage) on Sep 21, 2008 at 15:24 UTC
If you get `Bad hostname 'www.yahoo.com'`, then it sounds a bit like DNS isn't working on that box. Is DNS properly configured? -- Randal L. Schwartz, Perl hacker	[reply] [d/l]
Re^2: LWP running as cgi by Anonymous Monk on Sep 21, 2008 at 15:44 UTC
The box itself doesn't run a DNS. I didn't think it was required as running the script from the command line works (and I can use web browsers from it with no problem). Would there be a difference between command line vs cgi that required a DNS be installed locally?	[reply]
Re^3: LWP running as cgi by jethro (Monsignor) on Sep 21, 2008 at 16:01 UTC
The DNS server need not be on your machine, the question was whether DNS name resolution works at all. Since it works on the command line (on that machine) there is still the question whether maybe the web server can't access DNS. Security features like SELinux, SuSEs AppArmor or a chrooted apache environment might hinder the cgi script from accessing the DNS server or other web servers.	[reply]
Re^4: LWP running as cgi by betterworld (Curate) on Sep 21, 2008 at 16:21 UTC
Re^4: LWP running as cgi by elwoodblues (Novice) on Sep 22, 2008 at 10:35 UTC
Re^5: LWP running as cgi by betterworld (Curate) on Sep 22, 2008 at 23:10 UTC
Some notes below your chosen depth have not been shown here
Re: LWP running as cgi by oko1 (Deacon) on Sep 21, 2008 at 15:42 UTC
I recently wrote a post about this exact problem. As merlyn says, it sounds like a DNS problem: your system is not resolving 'yahoo.com' to an IP address. This may be on the networking side (e.g., you're not connected, or you don't have a valid DNS server defined) or it may be a problem in your code (in principle, that is; it works fine for me.) Assuming you've got a reasonable OS, you can test DNS with "host", "dig", or a bunch of other tools. -- "Language shapes the way we think, and determines what we can think about." -- B. L. Whorf	[reply]
Re^2: LWP running as cgi by Anonymous Monk on Sep 21, 2008 at 16:20 UTC
Thanks all, at least I now know what to look for. FYI it is Fedora 8, default install sudo -u apache dig http://www.yahoo.com works Interestingly, when I do: sudo -u apache ./lwptest.cgi it works, so I don't think it's related to the user permissions, pushing me back to the idea of paths.	[reply]
Re^3: LWP running as cgi by holli (Abbot) on Sep 21, 2008 at 17:47 UTC
You are using sudo, so you run this as admin. Of course that will not reveal any permission problems of the webserver-user. Do `su www-data #or whatever account the apache runs under` [download] and try above commands again. holli, /regexed monk/	[reply] [d/l]
Re^4: LWP running as cgi by elwoodblues (Novice) on Sep 21, 2008 at 18:07 UTC
Re: LWP running as cgi by jethro (Monsignor) on Sep 21, 2008 at 15:39 UTC
you might check that on the command line using `strace -f -o /tmp/bla perl <yourcgi> grep kerberos /tmp/bla grep usr/local /tmp/bla ...` [download]	[reply] [d/l]
Re: LWP running as cgi by kubrat (Scribe) on Sep 21, 2008 at 17:41 UTC
Try substituting www.yahoo.com with one of yahoo's ip addresses, if it works then you can be pretty certain that it is a DNS issue. I think, Fedora 8 comes with SElinux installed and configured, so that's most likely what is causing the issue.	[reply]