noniekimp has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to access a zip file in an ftp directory using LWP::UserAgent using $ua->credentials and am getting HTTP 401 Error: Unauthorized Access. I pasted the url directly into the browser, logged in with the same username and password, and it allowed me to save the document. I was able to get it to work properly with a text file.
$filename = 'file.zip'; # for example use LWP::UserAgent; use HTTP::Request::Common; my $url = "http://www.domain.com/$filename"; my $ua = new LWP::UserAgent(keep_alive=>1); $ua->credentials('www.domain.com:80', '', 'user', 'pass'); $request = GET $url; $response = $ua->request($request); print "response:" . $response->code . "\n";
I even tried changing to this syntax for the username and password:
$ua->credentials('www.domain.com:80', '', 'user'=>'pass');
Not sure what else to do here. Need help!

janitored by ybiC: Balanced <code> tags around codeblock, as per Monastery convention

Replies are listed 'Best First'.
Re: LWP Credentials Login Problem to Download Zip Files
by ikegami (Patriarch) on Aug 23, 2004 at 21:16 UTC

    I don't know how to use $ua->credentials, but I've used the following method in the past:

    use LWP::UserAgent; use HTTP::Request::Common; my $url = "http://www.domain.com/$filename"; my $ua = LWP::UserAgent->new(keep_alive=>1); my $headers = HTTP::Headers->new() $headers->authorization_basic('user', 'pass'); my $request = HTTP::Request(GET => $url, $headers); my $response = $ua->request($request); print "response:" . $response->code . "\n";

    I think using $ua->credentials ends up sending two requests to the server (the first being answered with a 401), whereas the methods I presented gets it right the first time.

    PS: Please put code in <code>..</code> tags in the future.

      It worked. I've gained access, but now I'm running into problems returning the file itself. I get the response of 200, which is great, but I would like it to prompt me to save the file. What am I doing wrong?
      use LWP::UserAgent; use HTTP::Request::Common; my $filename = 'file.zip'; # example my $url = "http://www.domain.com/$filename"; my $ua = LWP::UserAgent->new(keep_alive=>1); my $headers = HTTP::Headers->new(); $headers->authorization_basic('user', 'pass'); my $request = new HTTP::Request(GET => $url, $headers); my $response = $ua->request($request); print "response:" . $response->code . "<br>"; open(ZIP, ">$filename"); binmode ZIP; print ZIP $response->content; close(ZIP);

        I don't quite understand the question. That code works for me (although you might want to check if open() returns an error).

        Are you asking how to ask the user for a file name? You could use Perl/Tk for a dialog-based approach, or here's a unrefined console-based approach:

        print("Enter local filename, or just Enter to abort: "); my $local_filename = <STDIN>; chomp($local_filename); if ($local_filename ne '') { open ... }
      thanks for your help. sorry about all the repeating code. not intentional!
      Just wanted to say Super Search brought be here (and three other places) based on a very similar problem I was having and THIS did the trick for me! Thanks.
      This is what I added:
      open(ZIP, ">$filename"); binmode ZIP; print ZIP $response->content; close(ZIP); print "Content-Disposition: attachment; filename=$filename\n"; print "Content-type: application/zip\n\n";
      But for some reason the filename is just "download". Hmm! Close but no cigar. Need help!
        Oh! this is being executed in a CGI script? I don't know the answer. Try doing a search of this site, or start a new thread about this new problem so people who might know the answer notice it.
        I was running into problems trying to get the zip file to download. I was getting the prompt to download, but the file was coming up empty. The problem I was running into was that the connection to the browser is a stateless connection, and lost the credentials that was being used in getting the file in the first step. Had to save the file locally to a temp directory and create a direct link, (files are deleted after 24 hours) allowing people to download the file directly. This is what I ended up doing:
        $filename = 'test.zip'; # get file from external ftp use LWP::UserAgent; use HTTP::Request::Common; my $url = "http://www.domain.com/$filename"; my $ua = LWP::UserAgent->new(keep_alive=>1); my $headers = HTTP::Headers->new(); $headers->authorization_basic('user', 'pass'); my $request = new HTTP::Request(GET => $url, $headers); my $response = $ua->request($request); print "response:" . $response->code . "<br>"; # save file locally $filepath = "e:/myfolder/Data/$filename"; open(ZIP, ">$filepath"); binmode ZIP; print ZIP $response->content; close(ZIP); # provide link to download file print "Download file <a href="http://www.localftp.com/Data/$filename"> +here</a>";
        Don't think there was a solution to downloading the file via the a prompt.
Re: LWP Credentials Login Problem to Download Zip Files
by Aristotle (Chancellor) on Aug 23, 2004 at 21:15 UTC
    $ua->credentials( 'www.domain.com:80', '', 'user' => 'pass' );

    Are you sure that middle parameter should be empty? If it doesn't match the "realm" configured in the server for that location, LWP won't know it is supposed to use these credentials to authorize itself.


    In response to your wondering about the => arrow — it does nothing in your example. In fact, it's exactly the same as a comma. The difference is that this so-called "fat comma" lets you leave the left side unquoted, so you could say

    $ua->credentials( 'foo:80', 'bar', user => 'pass' ); # needs no quotes

    and get exactly the same result. There is no other difference, it's not magical in any way. It's just a bit of syntactic sugar to let you avoid quote noise when you want to. You can even use several of them in a row:

    $ua->credentials( 'foo:80', bar => user => 'pass' );

    But that quickly gets annoying to look at if you overdo it for no particular reason. It's nice to have when you want to make certain relationships stand out in the code, though. Sometimes, rarely, chaining several of them is actually sensible.

    It's just one those little things in Perl.

    Makeshifts last the longest.

Re: LWP Credentials Login Problem to Download Zip Files
by xorl (Deacon) on Aug 23, 2004 at 20:57 UTC
    Should't you be using Net::FTP if this is in an FTP directory?