true has asked for the wisdom of the Perl Monks concerning the following question:

Hi, Last night i wanted to use the LWP::UserAgent library on my webserver (Linux). I already had LWP::Simple installed, but when i tried to use UserAgent in my code my error logs said the library was not found. So i went into CPAN and installed the "LWP" library with no reported errors ("cpan> install LWP") LWP::UserAgent now runs in perl but all requests return blank content. The status message reports "OK 200", but there is no body content. (I tried the same request script on my home machine and it works fine). But now a larger problem has developed with LWP::Simple, if i use LWP::Simple to fetch any local domains (i.e. virtual hosts) they now return as empty content with no errors. Here's a test script i wrote which illustrates the problem... 
#!/usr/bin/perl
use strict;
use LWP::Simple;
print "fetching http://www.jtrue.com\n";
my $jfr = get "http://www.jtrue.com";
print "BEGIN FETCH______________\n";
print $jfr."\n";
print "END FETCH________________\n";

print "\n\n\nAnd now we fetch something not local...\n";
print " fetching http://www.google.com\n";
my $gfr = get "http://www.google.com";
$gfr =~ s/^(.{100}).*/$1\.\.\./s;
print "BEGIN FETCH______________\n";
print $gfr."\n";
print "END FETCH________________\n";

[download]

When i run this in a shell google returns fine, but jtrue.com (A domain on my server) returns blank. If i take this same script and put it on another server, both domain return fine.

Any help would be tremendous!

Thanks!

Just to clarify...

  • LWP::UserAgent NEVER returns content regardless of url.
  • LWP::Simple returns content ONLY if the domain is hosted elsewhere (not locally).

    P.S. Digs report no DNS errors.

    jtrue

    • Replies are listed 'Best First'.
    Re: CPAN LWP Install won't return local requests
    by sflitman (Hermit) on Dec 20, 2009 at 15:49 UTC
      The code works fine for me too, but I'm not local. It would be interesting to step through it with the Perl debugger and see if something odd is happening inside LWP::Simple, probably with name resolution which is happening locally. Are you running a virtual host Apache server or is there an entry for jtrue.com in your /etc/hosts file? That would explain the anomalous behavior.

      HTH,
      SSF

    		[reply]
        FYI, i run my own DNS and i am servering properly through named. You can poing, dig, and whois me from anywhere with no problems. You can even "get" my homepage with LWP::Simple from any other webserver.

        Here's some more data though i just uncovered...

        I download a local copy of LWP and renamed it JTLWP::Simple (sorry but i had to be sure i was using my own version)

        I looked at the "get" method inside and saw this... 

        $response->decoded_content

        [download]
        I changed it to this... 
        $response->content

        [download]
        And now i get a response from my LWP::Simple get requests. SO i guess this means my LWP library has a broken decoder?? ANyone else seen this or have any ideas?

        Thanks Monks!

        jtrue
    		[reply]
    [d/l]
    [select]
          Which version of LWP::UserAgent? Where is your LWP::UserAgent code? Did you turn on LWP::Debug?
    		[reply]

    Back to Seekers of Perl Wisdom