Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello natable Monks, Could you help me ? I would like to save periodically a text file from remote server in my local server directory. Unfortunately, I do not have LWP::Simple, and there is no chance I can use it (I am not authorized to modify Perl directory). ex. remote file http://www.remote.com/test.txt I would like to save it in my local directory e.g. /docs/test.txt Thanx in advance Jacek

Update: s/LWT/LWP/ in title and body, added link to CPAN. larsen

Replies are listed 'Best First'.
Re: replacement for LWT::Simple ?
by Biker (Priest) on May 13, 2002 at 15:32 UTC

    Is that LWP::Simple ?

    Try one of these:

    • Copy LWP::Simple to a directory where you can create files. (Your application directory?)
      • Prepend the directory to your @INC BEGIN{unshift(@INC,'directory');}
      • or
      • use lib 'directory';
    • Include the code of LWP::Simple.pm in your own script, some tweeking to make it a package in your own application and try it.
    • Use wget (Biker ducks and runs after having proposed a non-Perl solution ;-)


    Update: Doh! wil beat me to it. ;-)
    Everything went worng, just as foreseen.

(wil) Re: replacement for LWT::Simple ?
by wil (Priest) on May 13, 2002 at 15:26 UTC
    Here's a few options that you might consider.

    The first is that you can still download and install LWP::Simple but just install it into your local directory (unfortunately this might have to be a web visible directory for you). I've done this with my host once when they wanted me to test a package before installing it onto the main servers. Needless to say, I've moved host sine then ;-)

    The next option is to download LWP::Simple and extract the code from there and put it into your own stand alone script. This can be a good way to overcome this kind of problem, but I'd contact the module's author first or read up on the licence that come with it.

    Or your other option is to look into programs such as Wget which can be configured to retrieve files on from remote servers on a regular basis.

    Other than that, try looking around for a more flexible hosting solution.

    - wil
Re: replacement for LWP::Simple ?
by clintp (Curate) on May 13, 2002 at 19:35 UTC
    Do what everyone else here suggests (install LWP::Simple locally -- you can do it without root permissions it's just tedious) and use this only as a last resort. No warranties are implied. If you ask followup questions on how to use it I'll disavow all knowledge of you and label you a pest. :)

    But it will fetch web pages using only the base installation of Perl.

    use IO::Socket; use strict; use warnings; my $sock=new IO::Socket::INET( PeerAddr => 'www.foo.org', PeerPort => 80, Proto => 'tcp', Type => SOCK_STREAM); die "No socket" unless $sock; print $sock "GET http://www.foo.org 1.0\n\n"; my $answer=join('', <$sock>);
Re: replacement for LWP::Simple ?
by choocroot (Friar) on May 13, 2002 at 20:57 UTC
    >
    > Unfortunately, I do not have LWT::Simple, and there is no
    > chance I can use it (I am not authorized to modify Perl
    > directory).
    >

    You can install additional modules in your home directory without writing anything to the perl directory.

    Create a "perllib" directory in your home, grab and unpack the .tar.gz source of LWP and build it with:

    perl Makefile.PL PREFIX=$HOME/perllib LIB=$HOME/perllib make make install
    If you want to use CPAN, you can use it by adding 'PREFIX=/home/yourhome/perllib LIB=/home/yourhome/perllib' when CPAN configurator ask you for additionnal parameters for Makefile.PL
    Then you should be able to directly build and install the LWP module with:
    perl -MCPAN -e 'install LWP'
    So, now the module files should be in your "perllib" dir.
    Now you need to tell Perl that there is a new directory that contain modules by adding this directory to the PERL5LIB variable:
    export PERL5LIB=$PERL5LIB:$HOME/perllib
    And it should work ...

    (perlmodinstall may also help)
Re: replacement for LWP::Simple ?
by Anonymous Monk on May 13, 2002 at 22:45 UTC
    Thank you guys, but I am just a beginner ;-) so please be patient. I copied LWP module to my local /cust-bin/ directory. I tried a simple script:
    #!/usr/local/bin/perl use lib './LWP'; use LWP::Simple; $myDocument = get "http://www.ecst.csuchico.edu/"; print "Content-type: text/html\n\n", $myDocument;
    I got a blank page rather then http://www.ecst.csuchico.edu/
    But it does not work.
    What should I do ?

    Jacek

    Update: added <code> tags, minor HTML adjustements. larsen

      What should I do ?

      Check your error_log(s) to see what went wrong. Your error_log usually gives you an accurate reading of what went wrong, or a preety good idea of where to start looking.

      It is most probably an error relating to the LWP module you just installed. Try Sweeper's suggestion and see where that leads you.

      - wil

      Copy Simple.pm to the /cust-bin/LWP directory. (Create if necessary.)

      use lib '/cust-bin'; use LWP::Simple;

      Everything went worng, just as foreseen.

      Should you write use lib './LWP'; or use lib '.'; ? The first one will seek ./LWP/LWP/Simple.pm, which may not be what you intended.
      On the other hand, you can write both lines, it won't hurt.
      On the gripping hand I have not tested it, I may be wholly wrong.
Re: replacement for LWP::Simple ?
by Anonymous Monk on May 14, 2002 at 10:07 UTC
    Hi, Running the following script with debugger:
    #!/usr/local/bin/perl -w use lib '.'; use LWP::Simple; $myDocument = get "http://www.ecst.csuchico.edu/"; print "Content-type: text/html\n\n", $myDocument;

    I received

    Useless use of a variable in void context at /docs/cust-bin/test.cgi
    line 6. Content-type: text/html
    Jacek

      Did you cut-and-paste that script when you posted it above? When I run it, it works fine. It is a 5 line script, with the last statement beginning on line 5, and continued onto line 6; errors in the part of the statement on line 6 would be reported as line 5. If you mis-keyed the last line with a semi-colon in place of the comma, then you would get the exact error you described.

      # Bad semi-colon here..............* print "Content-type: text/html\n\n"; $myDocument;
Re: replacement for LWP::Simple ?
by Anonymous Monk on May 14, 2002 at 15:42 UTC
    I run script
    #!/usr/local/bin/perl -w use lib '.'; use LWP::Simple; $myDocument = get "http://www.ecst.csuchico.edu/"; print "Content-type: text/html\n\n"; $myDocument;
    still error message appears:
    Useless use of a variable in void context at /docs/cust-bin/test.cgi l +ine 8. Content-type: text/html
    so there is an error while processing --> $myDocument;

    Jacek

      Correct! I was asserting that you had tested this:
      print "Content-type: text/html\n\n"; $myDocument;
      but posted this:
      print "Content-type: text/html\n\n", $myDocument;

      A comma would mean that $myDocument is the second argument to the print statement, and should work. A semi-colon would mean that $myDocument was a statement in itself, which is nonsensical, and will give you the error message you posted.

      I think that you actually tested using a semi-colon twice, and happened to mis-key the semi-colon when you posted the first time, changing it into a comma and fixing your own problem! (Or maybe larsen fixed your error, and you cut-and-pasted his correction ?)

      Can you test your program again, making sure you have only a comma between "Content-type: text/html\n\n" and $myDocument ?

Re: replacement for LWP::Simple ?
by Anonymous Monk on May 14, 2002 at 19:07 UTC
    I tried with comma, and I got:
    Use of uninitialized value at /docs/cust-bin/test.cgi line 6. Content- +type: text/html
    so this time the error points to
    print "Content-type: text/html\n\n",
    Maybe the server somehow prevents LWP modules being executed ?
    Jacek

      If you get the error Use of uninitialized value at ..., this always means that you tried to print (or otherwise use) something which had no value assigned to it (or the value undef). This has nothing to do with modules.

      My guess is, that your get "http://www.ecst.csuchico.edu/"; fails for some reason (we won't find the reason as LWP::Simple is also simple in its error handling). So you could either double-check your url, or add a conditional to your code like the following :

      my $myDocument = get "http://www.ecst.csuchico.edu/"; my $result; if ($myDocument) { # we got a response $result = $myDocument; } else { # Something went wrong $result = "<tt>There was a problem somewhere and I didn't get an ans +wer.</tt>"; }; print "Content-type: text/html\n\n"; print $result;
      perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web
Re: replacement for LWP::Simple ?
by Anonymous Monk on May 14, 2002 at 22:47 UTC
    I tried:
    #!/usr/local/bin/perl -w my $myDocument; my $result; use lib '.'; use LWP::Simple; $myDocument = get "http://www.ecst.csuchico.edu/"; if ($myDocument) { # we got a response $result = $myDocument; } else { # Something went wrong $result = "<tt>There was a problem somewhere and I didn't get an ans +wer.</tt>"; }; print "Content-type: text/html\n\n"; print $result;
    And I got a message:
    There was a problem somewhere and I didn't get an answer.
    Jacek

      Since your code works correctly for us, but not for you, you will have to do the debugging. You may need to tell LWP to use a proxy; check your web-browser to see whether it has a proxy configured. Here are some documents on LWP proxy use and debugging:

      Also, from your "Content-type: text/html\n\n" statement, I take it that you are testing this program as a CGI script on a web server. Can you test it from the command line instead? If so, you will be able to see error output that would have been hidden by the web server. Get as much working from the command line as you can, and only then turn it into a CGI; much aching of the head will be avoided.

Re: replacement for LWP::Simple ?
by Anonymous Monk on May 15, 2002 at 18:44 UTC
    Thank you everybody for help.Maybe the problem is with a Perl.
    My server has Perl ver. 5.003_5, and could you advise me
    what and which ver. of modules should I copy to my directory
    eg. LWP, URI, UserAgent and others ?
    Jacek