in reply to Help with Time

I am going to suggest a little different approach. Since downloading is an activity that is very prone to failure, I would maintain a list of downloads that need to be completed and break this operation into two processes: one would scan the list of downloads that need to be completed and then attempt to complete them; the other would add a new download task to the list every night (i.e. downloading last night's games.) The advantage of this is that in the event of a download failure, you can just run the first process again, and it will try to complete whatever download tasks failed.

The implementation can be very simple. Maintain two directories, unfinished/ and completed/. A download task can be represented by a file in unfinished/ whose name is the date of the desired download. When the download is complete, simply rename the file from the unfinished/ directory to the completed/ directory.

To add a download task, just create a new file in unfinished/ with the right name. This can be done via a cron job or even manually to redo a previous date.

Some more details...

Let's assume that the files in the unfinished/ directory have the form yyyy-mm-dd. Here's how to process each file: 

opendir(D, "unfinished/") or die "unable to open unfinished/: $!\n";
my @files = readdir(D);
closedir(D);
for (@files) {
  next unless ($file =~ m/^(\d\d\d\d)-(\d\d)-(\d\d)$/);
  my ($year, $month, $day) = ($1, $2, $3);
  eval {
      ... process download for $year/$month/$day ...
      ... in case of an error, raise an exception with die ...
  };
  if ($@) {
    warn "downloading for $year/$month/$date failed: $@\n";
  } else {
    rename("unfinished/$file", "completed/$file");
  }
}

[download]
To create a new download task for yesterday's games: 
my @t = localtime(time-86400);
my $y = $t[5]+1900;
my $m = $t[4]+1;
my $d = $t[3];
my $file = sprintf("%04d-%02d-%02d", $y, $m, $d);
open(F, ">", "unfinished/$file");
close(F);

[download]

Replies are listed 'Best First'.
Re^2: Help with Time
by TheAnswer1313 (Initiate) on Apr 20, 2008 at 16:28 UTC
    Thanks. I might consider doing it that way. I don't have any experience with perl tho. I didn't create the script, rather im modifying an existing one. The download files are very small and I havent experienced any problems yet. What if I just wanted to change the existing one instead of having two separate tasks? Understanding I might have some downloading issues, is there still a way I can do this?
[reply]
      As apl suggests, there are several modules on CPAN to manipulate dates and times. One is DateTime which you could use like this: 
      use DateTime;

my $d = new DateTime(year => 2008, month => 3, day => 31);
my $yesterday = DateTime->today()->add(days => -1);
while (1) {
  my $year = $d->year;
  my $month = $d->month;
  my $day => $d->day;
  ... download file for $year, $month, $day ...
  last if ($d->ymd('/') eq $yesterday->ymd('/'));
  $d->add(days => 1);
}

      [download]
[reply]
[d/l]
[select]
        Thanks I'll give that a try. The one its using right now is Time::Local i believe.
[reply]

In Section Seekers of Perl Wisdom