The implementation can be very simple. Maintain two directories, unfinished/ and completed/. A download task can be represented by a file in unfinished/ whose name is the date of the desired download. When the download is complete, simply rename the file from the unfinished/ directory to the completed/ directory.

To add a download task, just create a new file in unfinished/ with the right name. This can be done via a cron job or even manually to redo a previous date.

Some more details...

Thanks. I might consider doing it that way. I don't have any experience with perl tho. I didn't create the script, rather im modifying an existing one. The download files are very small and I havent experienced any problems yet. What if I just wanted to change the existing one instead of having two separate tasks? Understanding I might have some downloading issues, is there still a way I can do this?

As apl suggests, there are several modules on CPAN to manipulate dates and times. One is DateTime which you could use like this:

use DateTime;

my $d = new DateTime(year => 2008, month => 3, day => 31);
my $yesterday = DateTime->today()->add(days => -1);
while (1) {
  my $year = $d->year;
  my $month = $d->month;
  my $day => $d->day;
  ... download file for $year, $month, $day ...
  last if ($d->ymd('/') eq $yesterday->ymd('/'));
  $d->add(days => 1);
}
[download]

Here are a couple of hints:

• You need only change the line that creates the $start time.
• Ideally, you'd want to start one day before today, right? So take a look at what the for loop is doing.
• perldoc -f time
• perldoc -f timelocal

I offered hints rather than the answer because I believe that you'll want to learn perl quickly so you can edit these scripts you've apparently inherited. And there's no better way than diving in and investigating the code. Of course, a couple of hints here and there about what to look at first can be a real timesaver.... ;^) If you *really* just want the answer, let me know....

I really want the answer lol.

I thought I could just change the line for ($t = $start; $t < $now; $t += 60*60*24) { to for ($t = $start; $t = $now; $t += 60*60*24) {

If I just change the $start time tho.....wouldnt I still have to change the $now line as well? I dont know.....Im confused. I just wanted this file to spider my baseball data for me to save time......since manually downloading the files would take forever lol.

TheAnswer1313:

OK ... the first lines of your code are choosing the first day to download, 3/31/08 in this case:

# get all important files from MLB.com, 3/31/08 through yesterday
#$start = timelocal(0,0,0,31,2,108);
$start = timelocal(0,0,0,31,2,108);
[download]

Read up on the timelocal function if you need more explanation of this part. In fact, read up on timelocal before continuing, or the next bit won't make sense.

You want to start downloading "yesterdays" file, instead of that hardcoded date. So you need to figure out how to get yesterday's date in the same format that timelocal is going to give it to you (so you don't need to change the code).

It turns out that timelocal returns the number of seconds since XXXXX. The time function returns the *current* date/time also in seconds since XXXXX. So time almost gives you what you want. Since there are 24 * 60 * 60 seconds in a day, if you subtract that from time, you'll get the same time yesterday. So you could change the lines above to:

# Start downloading from yesterday
$start = time - 24*60*60;
[download]

The rest of the code (including $now) should be just fine. I hope this helps!

...roboticus

Note: The XXXXX is a specific date, but it's unimportant, so I didn't bother to specify.

Update: Added bit about $now.