NOAA updates its weather satellite images on the GOES website every 30 minutes, with images in infrared, visible light, and water vapor wavelengths; for both the eastern and western halves of the US. The script below grabs those images and saves them to a local file with a timestamp. Could prove useful for doing your own weather forecasting or whatever.... 
    use strict;
    use LWP::Simple;
    use Time::Tiny;

    my $image;
    my $url;
    my %images = (
    "eastconusir" => "http://www.goes.noaa.gov/GIFS/ECIR.JPG",
    "eastconusvis" => "http://www.goes.noaa.gov/GIFS/ECVS.JPG",
    "eastconuswv" => "http://www.goes.noaa.gov/GIFS/ECWV.JPG",
    "westconusir" => "http://www.goes.noaa.gov/GIFS/WCIR.JPG",
    "westconusvis" => "http://www.goes.noaa.gov/GIFS/WCVS.JPG",
    "westconusvwv" => "http://www.goes.noaa.gov/GIFS/WCWV.JPG",
    );

    my $current_time = Time::Tiny->now;
    my $timestring = $current_time->hour.$current_time->minute.$curren
+t_time->second;
    print $timestring."\n";

    foreach my $key (keys %images) {
    print $key."\n";
    print $images{$key}."\n";
    my $status = getstore($images{$key},$key.$timestring.".jpg");
    print $status."\n" ;
    };

[download]

Replies are listed 'Best First'.
Re: Automated extraction of NOAA weather satellite images
by graff (Chancellor) on May 30, 2007 at 02:40 UTC
    I had never heard of Time::Tiny before... If I understand its man page correctly, I think your code would create variable-length file names that might be hard to interpret reliably -- and might not be unique over the course of a day. That is, when you do: 
    my $timestring = $current_time->hour.$current_time->minute.$current_ti
+me->second;

    [download]
    the time-stamp portion of each file name will range from "*.000.jpg" at midnight to "*.235959.jpg" at one second before midnight, with potential file-name collisions among sets of times like 01:23:45, 12:03:45 and 12:34:05 (which all would end up as "*.12345.jpg"), for example.

    Personally, I'd use the POSIX "strftime" function to set up fixed-width (zero-padded) strings for the file names -- and while I'm at it, might as well throw in the "ymd" date as well. Also, I think I'd rather use less repetition in the literal strings (looks like there was an inconsistency in your file names: an extra "v" in "westconusvwv"). 

    use strict;
use LWP::Simple;
use POSIX;

my %image_url;
for my $e_w ( qw/eastconus westconus/ ) {
    for my $typ ( qw/ir vs wv/ ) {
        $image_url{$e_w.$typ} = "http://www.goes.noaa.gov/GIFS/" .
             uc(substr($e_w,0,1)) . 'C' . $typ . '.JPG';
    }
}
my $timestring = strftime( "%Y%m%d_%H%M%S", localtime );
print $timestring;

for my $img ( keys %image_url ) {
    my $status = getstore( $image_url{$img}, join( '.', $img, $timestr
+ing, 'jpg' ));
    print join( "\n", $img, $image_url{$img}, $status, "\n" );
}

    [download]
    For that matter, it would be even simpler if I decided that the original four-letter NOAA file names were good enough for me...

    (updated the code to add a missing close-paren at line 16. Thanks, zentara!)

[reply]
[d/l]
[select]
      Thanks for the feedback. I was looking at the POSIX module for including a timestamp in the saved filenames, but wanted to get a script up and running quickly and having never used the "strftime" function you mention Time::Tiny filled the bill. I agree with the potential problems you point out that the timestamps in my code have... I have updated my code to use the "strftime" function and the filenames now are very precise(for example "eastconusir20070530_074348.jpg"): 
      #NOAA updates the images on the GOES website every 30 minutes

use strict;
use LWP::Simple;
use POSIX;


my $image;
my $url;
my %images = (
 "eastconusir"  => "http://www.goes.noaa.gov/GIFS/ECIR.JPG",
 "eastconusvis" => "http://www.goes.noaa.gov/GIFS/ECVS.JPG",
 "eastconuswv"  => "http://www.goes.noaa.gov/GIFS/ECWV.JPG",
 "westconusir"  => "http://www.goes.noaa.gov/GIFS/WCIR.JPG",
 "westconusvis" => "http://www.goes.noaa.gov/GIFS/WCVS.JPG",
 "westconusvwv" => "http://www.goes.noaa.gov/GIFS/WCWV.JPG",
        );

my $timestring = strftime( "%Y%m%d_%H%M%S", localtime );
print $timestring."\n";

foreach my $key (keys %images) {
    print $key."\n";
    print $images{$key}."\n";
    my $status = getstore($images{$key},$key.$timestring.".jpg");
        print $status."\n" ;
    };

      [download]


      See my Standard Code Disclaimer
[reply]
[d/l]
Re: Automated extraction of NOAA weather satellite images
by jasonk (Parson) on May 30, 2007 at 13:37 UTC

    As it happens, I wrote a very similar script a couple of weeks ago. Rather than using the current time though, mine builds a timestamp from the Last-Modified header of the response, so the timestamp is the last time the image was updated, and you can run it as often as you want without downloading duplicate images.

    #!/usr/local/bin/perl -w
##################
# Copyright 2007 Jason Kohles (JSK), <email@jasonkohles.com>
##################
use strict;
use warnings;
use LWP::Simple qw( getstore head );
use POSIX 'strftime';

#chdir( "/home/weather/noaa-goes-images" ) or die "Cannot chdir: $!";

for my $loc ( qw( EC WC ) ) {
    for my $type ( qw( VS IR WV ) ) {
        my $url = "http://www.goes.noaa.gov/GIFS/".$loc.$type.".JPG";
        my $epoch = ( head( $url ) )[ 2 ] || next;
        my $time = strftime( '%F-%T', gmtime( $epoch ) );
        my $file = lc( join( '-', $loc, $type, $time ).'.jpg' );
        next if -f $file;
        print "Retrieving $url to $file\n";
        getstore( $url, $file );
    }
}

    [download]
    We're not surrounded, we're in a target-rich environment!
[reply]
[d/l]
      It's a good idea to timestamp the images with Greenwich Mean Time since that's what NOAA uses: I updated my script with your modified timestamp.

      #NOAA updates the images on the GOES website every 30 minutes

use strict;
use LWP::Simple;
use POSIX;


my $image;
my $url;
my %images = (
 "eastconusir"  => "http://www.goes.noaa.gov/GIFS/ECIR.JPG",
 "eastconusvis" => "http://www.goes.noaa.gov/GIFS/ECVS.JPG",
 "eastconuswv"  => "http://www.goes.noaa.gov/GIFS/ECWV.JPG",
 "westconusir"  => "http://www.goes.noaa.gov/GIFS/WCIR.JPG",
 "westconusvis" => "http://www.goes.noaa.gov/GIFS/WCVS.JPG",
 "westconusvwv" => "http://www.goes.noaa.gov/GIFS/WCWV.JPG",
        );



foreach my $key (keys %images) {
    print $key."\n";
    my $epoch = ( head( $images{$key} ) )[ 2 ] || next;
    my $timestring = strftime( "%Y%m%d_%H%M%S", gmtime( $epoch ) );
    print $timestring."\n";
    print $images{$key}."\n";
    my $status = getstore($images{$key},$key.$timestring.".jpg");
        print $status."\n" ;
    };

      [download]
      See my Standard Code Disclaimer
[reply]
[d/l]

Back to Cool Uses for Perl