Trouble downloading .KMZ files with www-mechanize

cbouwkamp has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to download a file every 45 minutes from a site. It is a .kmz file which is an overlay you can open in google earth to view. Problem is that the method I am using seems to now work correctly,but gives me a corrupted file. Any help here is my code:

#!/usr/bin/perl -w 
use strict; 
use LWP; 
use WWW::Mechanize; 
my $now_string = localtime; 
my $mech = WWW::Mechanize->new(); 
my $filename = join('_', split(/\W+/, $now_string, -1)); 
$mech->credentials( '*********' , '***********'); 
$mech->get('http://datawww2.wxc.com/kml/echo/MESH_Max_180min/') or die
+ "Error: failed to load the web page"; 
#print $mech->response()->content(); 
$mech->follow_link(n => 8) or die "Error: failed to download content";
+ 
$mech->save_content("$filename.kmz");
[download]

I am wondering if this is an okay way to obtain those types of files or not? I know the file is corrupted because when I click the link myself and download the file it works fine. The file is a kmz file which is basically an image overlay of weather that can be used on google earth or other web maps. When I open them up in notepad they are very jarbled symbols but look relatively the same. i dont know about the unicode flavor.

Comment on Trouble downloading .KMZ files with www-mechanize Download Code

Replies are listed 'Best First'.
Re: Trouble downloading files by desemondo (Hermit) on Jul 08, 2010 at 03:38 UTC
Welcome cbouwkamp, Please have a read the following as your question is lacking detail (and appears to be lacking effort which we despise ) : How do I compose an effective node title? How do I post a question effectively? How do you know the file is corrupted? If the file is plain text, what does it look like if viewed in notepad? Do you have a file that works correctly? - If so, what is different between it and the corrupted file when you compare them? Does the file need to be saved in some flavour of Unicode? Update: I am wondering if this is an okay way to obtain those types of files or not As far as I'm aware that should be fine... Since the files are not plain text, get hold of a good file comparison tool. I quite like winmerge but there are plenty of others. Use it to compare the same file, 1 that was retrieved using your script and the other using your browser. It should highlight exactly what is different between the two files. Update2: Have you tried using this `$mech->get( $url, ':content_file' => $filename );` to retrieve the file? www::Mech examples	[reply] [d/l]
Re: Trouble downloading .KMZ files with www-mechanize by Sinistral (Monsignor) on Jul 08, 2010 at 12:35 UTC
You need to consult a good definition of KML vs. KMZ as well as the KML tutorial and KML documentation. That being said, the reason your files looks "garbled" in notepad is that KMZ files are zipped files to conserve space (using the same zip algorithm that you can undo using Windows open zipped folder, 7-Zip, Winzip, and so forth). You don't want to use Winmerge to compare two KMZ files, because they are binary files. Unzip the files using your favorite zip program and then compare. Or, if you want to test directly, use WWW::Mechanize to retrieve the file, copy it to your local PC, and then attempt to load it using Google Earth. If the overlay works, then WWW::Mechanize is doing the right thing (which I'm pretty confident it is). Update I almost forgot to mention, KML files by default are XML encoded using UTF-8	[reply]
Re: Trouble downloading .KMZ files with www-mechanize by ww (Archbishop) on Jul 08, 2010 at 12:53 UTC
Further re local practice: Don't change the content of a node without also posting notice of the changes. Your revisions go a long way toward answering desemondo's advice and questions, but could mislead later readers about the value of the response. In other cases, unannounced changes can make a thread unintelligible.	[reply]