Seeking help on a backup script

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks,

I am trying to automate backing up a database and a few directories/files from a windows server to a backup server (linux). I intend to use perl because ActivePerl is already available on the windows server.

I am familiar with perl and not looking for the actual solution. I am seeking general guidance on the correct approach to solving the problem.

Currently, my high level design looks like this:

Dump the database to a file in $BACKUP dir using the vendor provided dump utility.
Copy other files to be backed up to the $BACKUP dir
Compress the contents of $BACKUP dir
Copy the compressed file to the remote backup server (ftp?)
Keep only the last N (3) backups on the server and delete older backups

I am having trouble with the last step above. Ideally, I want to call the backup script weekly or bi-weekly. The compressed filename will probably include the date and time to identify when the backup was made. I need some pointers on how to determine the older than N backup files and delete them.

Any help/pointers/suggestions (including that I am totally on the wrong track, there are better ways of doing this) is appreciated.

Regards, Anand Mathur

Comment on Seeking help on a backup script

Replies are listed 'Best First'.
Re: Seeking help on a backup script by holli (Abbot) on Aug 30, 2006 at 14:09 UTC
Chose a name scheme like BACKUP-$DATE.tar.gz, create a directory listing and sort it newest first. Then the first three elements in the array are to keep, the others to delete. holli, /regexed monk/	[reply]
Re: Seeking help on a backup script by nimdokk (Vicar) on Aug 30, 2006 at 13:49 UTC
For 5), you might want to look at the file test operators. Perhaps even storing the filenames of already processed files in an array that is dumped into a file that can be read in when the script runs (something like an external config file). Data::Dumper can help with this. This is the solution I use for some backups that I run on our systems. May not be perfect or elegant, but works for me. I'm sure others might have better solutions.	[reply]
Re: Seeking help on a backup script by cdarke (Prior) on Aug 30, 2006 at 15:02 UTC
The date/time stamp on the archive file may be enough, as holi remarked. I find it useful to store the date, preferrably as part of the file name (as nmdokk suggested) since copying can destroy the original date. Personally I always use the number of seconds since epoc, rather than a "display" date format. It is then much easier to compare with other dates in the same format. This is the same format used by most file systems, and is returned by stat. You can also get a similar number from time, but don't mix them.	[reply]
Re^2: Seeking help on a backup script by satchm0h (Beadle) on Aug 30, 2006 at 19:16 UTC
Given that you are going from a windows box I'm not sure if you ahve access to epoch time. You can always use the localtime values to build up A YYYYMMDDHHMMSS type of date string as well. And I third holli's suggestion of encoding the date into the filename and simply sorting based on the filename. If you want to go the extra mile you could leverage something like rsync to do incrementals . ...stumbling down the path toward enlightenment	[reply]
Re^3: Seeking help on a backup script by Anonymous Monk on Aug 31, 2006 at 16:15 UTC
Dear monks, Thank you for your ideas/suggestions. I am going to encode the date into the backup filename. It will be easy to determine the older files by sorting on the filenames. Additionally, the filenames will automatically include information on when the backup was made. So, the site admin can use a simple directory listing to check on the backup status. Kind Regards, Anand Mathur	[reply]