LWP Simple - getstore function

tanger has asked for the wisdom of the Perl Monks concerning the following question:

hi-

I am trying to save a image from a website by using the LWP Simple get store function. However, when being saved the image file is saved in ASCII format and not BINARY, leaving the picture to be displayed wrong. I researched on how I can do this and I think it invovles using binmode??

previous code:

my $image_name = "heybuddy.jpg";
my $image_full = "/home/temp/$image_name"; #image full path to the tem
+p file
my $status = getstore($video_src, $image_full); #uses getstore to retr
+ieve image and to be stored in the temp path.
die "Error $status on $video_src" unless is_success($status);
[download]

The above code correctly stores the file retrieved from the website, but now I need to convert the temp file to a binary file and store it in my image dir and not temp dir.

So I tried adding this right below the above coding:

open(IN, ">/home/images/$image_name") or die "Can't open: $!";        
+     
binmode IN;
    while (<$image_full>) {
    print IN;
    }
    close IN or die "Can't close: $!";
}
[download]

This code does save the image name to my image directory (home/images) but it comes out to 0 kilobytes.
Ahhhh any help will be appreciated.

tanger

Comment on LWP Simple - getstore function Select or Download Code

Replies are listed 'Best First'.
Re: LWP Simple - getstore function by BrowserUk (Patriarch) on Oct 31, 2004 at 05:38 UTC
If the file is written without binmode, it will have been corrupted and will likely be irrecoverable. Simply re-opening the file and stripping newlines is unlikely to work as the binary file may have containd some binary data which looks like newlines, and you would be stripping these too. getstore doesn't allow you to pass in an open filehandle, but internally, getstore is implemented using `LWP::UserAgent::request( url, $file );` and one of it's options is to allow the second parameter to be a callback address (See: LWP::USerAgent::Get() POD; ':content_cb' parameter.). This allows a simple work around of passing in an anonymous sub that prints the content to a pre-opened, binmode'd output file: `#! perl -slw use strict; use LWP::Simple; open my $fh, '> :raw', $ARGV[ 1 ] or die $!; my $code = getstore( $ARGV[ 0 ], sub { print $fh $_[ 0 ]; } ); close $fh; print "rc=$code";` [download] Note: I've used :raw', rather than binmode, but the effect is the same. Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "Think for yourself!" - Abigail "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon	[reply] [d/l] [select]
Re: LWP Simple - getstore function by bart (Canon) on Oct 31, 2004 at 07:08 UTC
Your concerrn is unfounded, getstore() retrieves files from URLs in binary mode just fine. Besides, if it had been downloaded in text mode, it would be corrupted beyond repair. Just download the file to its final destination, and it'll be fine. Or: use a temporary file name to download it to, and use rename to give it its final name, or even move the file to another directory, as long as it remains on the same disk.	[reply]
Re^2: LWP Simple - getstore function by tanger (Scribe) on Oct 31, 2004 at 07:19 UTC
Ahh...thats what i thought. On the google group forums I read several posts telling how getstore retrieves binary finals just fine. That is why an hour and a half later I'm still sitting here trying to figure this out. I just tried your renaming method and no such luck. Image still comes out in Ascii format with the image hash text. :(	[reply]
Re^3: LWP Simple - getstore function by bart (Canon) on Oct 31, 2004 at 07:21 UTC
Then it must be the server that sends it as text... or maybe the file is already corrupt on the server. I don't think you can see it properly directly in a browser either, can you? If, countrary to my expectations, you can, then check the MIME type (= content-type) the server says it is. It might say it's text/something.	[reply]
Re: LWP Simple - getstore function by Zaxo (Archbishop) on Oct 31, 2004 at 05:32 UTC
You are trying to use the file name as a handle to a file which is read out to the end. Try File::Copy. After Compline, Zaxo	[reply]
Re^2: LWP Simple - getstore function by tanger (Scribe) on Oct 31, 2004 at 05:53 UTC
hi- ty, I tried that but no luck :( It copies the ascii image file to the new dir which leaves the same result. I think I need to figure out how to convert it to binary?? or download it in binary mode?	[reply]
Re: LWP Simple - getstore function by Cody Pendant (Prior) on Oct 31, 2004 at 10:19 UTC
I say, give us the URL and let us try for you. Just a thought, but porn sites? Sometimes you can't download the images unless the REFERRER is also in their domain, and you get an HTML file which is an error message instead. ($_='kkvvttuubbooppuuiiffssqqffssmmiibbddllffss') =~y~b-v~a-z~s; print	[reply]
Re: LWP Simple - getstore function by allyc (Scribe) on Oct 31, 2004 at 13:50 UTC
I use the GetStore command to pull pictures of the web and save them. All I use is `use LWP::Simple; getstore("http://path.to.web/image.jpg","/path/to/myimage.jpg");` [download] Works like a treat on Images and Text/HTML files. getstore seems to work out the file handle's and bin mode its self. Al	[reply] [d/l]
Re^2: LWP Simple - getstore function by tanger (Scribe) on Oct 31, 2004 at 16:57 UTC
Hi ahhhhh! it was the server that had it all along. I just woke up today mourning with a fresher focus and relized i should replace my url string with any other test url. It worked so I narrowed the problem down to be that the image I was trying to retrieve before was off a server that has some sort of image redirection/protection?? I ended up getting the image from the place I wanted too but I had too add the full url I got from parsing ex. www.path.to.web/image.jpg?qlt=75&wid=175&cvt=jpg Instead I took that url and parsed it so I didn't have the extra parameters being passed. I figured that was the right way to retrieve that actual image file and OH BOY was I WRONG!!!! That cost me about a 2 hr headache sitting at the comp researching last night :((((. I guess last night I wasn't quick enough to try to narrow in on that problem, maybe being because I read so many getstore examples and how they just worked fine. I'm on a machine I just got on so In the back of my head I was hoping it was a server setting/module version or something that didn't make getstore working properly. thank you all for trying to help me Your advice and tips gained me knowledge on this topic and helped me figure this out. I'll be sure not to run into this type of problem again. tanger	[reply]