eric256 has asked for the wisdom of the Perl Monks concerning the following question:

I am building a networked backup system that will allow remote offices to backup to a central server. I was planning on having each client zip up just the changed files and then send that zip file to the server, which would then merge the new zip in with the existing zip, creating a kind of incremental backup. I can' figure out a way to merge two zip files together without unzipping and re-zipping the files together, which could get to be a mess sense the zip files will contain directory information.

Does any one have a clue how to go about merging zip files? or is there a better solution I’m just not seeing?

thanks!

Replies are listed 'Best First'.
Re: Combining or Merging 2 zip files.
by hardburn (Abbot) on Jun 05, 2003 at 16:11 UTC

    Look under "Add a directory/tree to a Zip" under Archive::Zip. Don't know about adding individual files, but this should get you started.

    ----
    I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
    -- Schemer

    Note: All code is untested, unless otherwise stated

      I understand how to add files to the zip and even how to manage sending it over the network. I just can't figure out how to take two seperate zip files, and combine them, overwriteing the old files in the main zip with the new files in the updated zip.

        For that, I think you'll have to unzip at least the updated zip. Unless you want to mess around with the guts of Zip files.

        ----
        I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
        -- Schemer

        Note: All code is untested, unless otherwise stated

Rsync for Incremental Backup
by Arguile (Hermit) on Jun 05, 2003 at 16:43 UTC

    Rsync.

    Use with the Perl interface File::Rsync. With it you can transfer only the differences between files, keeping bandwith usage much lower. There are also over the wire compression options and bandwidth throttling to further reduce/control it.

Re: Combining or Merging 2 zip files.
by freddo411 (Chaplain) on Jun 05, 2003 at 17:39 UTC
    If you decide you don't want to do all the heavy lifting in your perl code you might take a look at using Unison. I found it to be a nice, cross platform tool for providing automatic "backup" to remote sites over a network.

    Also, for a good concise read that includes lots of excellent background information look here. You'll find pointers to perl scripts for backup including one that I modified and used.

    Cheers

    -------------------------------------
    Nothing is too wonderful to be true
    -- Michael Faraday

Re: Combining or Merging 2 zip files.
by Dr. Mu (Hermit) on Jun 06, 2003 at 04:45 UTC
    Following is a CGI program I use to do just what you're describing. The file being uploaded is a zip file, whose members may or may not exist in the backup archive. Those members which exist, but with a different CRC, are updated; those which don't, simply added. I've left out some authorization code, which you should come up with to fit your situation. (You don't want malicious users backing up their files on your system!) I hope it helps. (Sorry 'bout the indents. SOPW seems to be a tab mangler.)
    #!/usr/bin/perl -w use strict; use Archive::Zip; use CGI; use CGI::Carp qw(fatalsToBrowser); use Digest::MD5 qw(md5_base64); my $BackupPath = '/myserver/mybackupdir'; my $InpFile = $query->param('Backup'); undef $/; my $InpData = <$InpFile>; my $len = length($InpData); print "Content-type: text/plain\n\n"; my $NewFile; if ($len) { $NewFile = rename("$BackupPath/remote.zip", "$BackupPath/remote.ba +k") ? 'send.zip' : 'remote.zip'; open BAK, ">$BackupPath/$NewFile" or die "Can't open send.zip to wri +te: $!"; print BAK $InpData; close BAK; print "Received: $InpFile. Saved as: $NewFile. Length: $len bytes\n" +; } exit if $NewFile eq 'remote.zip'; my $local = Archive::Zip->new("$BackupPath/send.zip") or die "Can't ac +cess send.zip"; my @local_files = $local->members(); my $remote = Archive::Zip->new("$BackupPath/remote.bak") or die "Can't + access remote.bak"; my @remote_files = $remote->members(); my %remote_index; foreach (@remote_files) { $remote_index{$_->fileName()} = $_->crc32String() } foreach (@local_files) { my $name = $_->fileName(); print "$name: "; if (exists $remote_index{$name}) { if ($remote_index{$name} eq $_->crc32String()) { print "Exists unchanged.\n"; next } else { print "Needs updating.\n"; $remote->replaceMember($name, $_) } } else { print "Doesn't yet exist.\n"; $remote->addMember($_) } } $remote->writeToFileNamed("$BackupPath/remote.zip") && die "Error writ +ing remote.zip."; unlink "$BackupPath/remote.bak"; #Or not. I had to to keep from going +over my host's storage limit. print "*** Zip file updated. ***\n";