Zip files while copying

calsaint has asked for the wisdom of the Perl Monks concerning the following question:

I am writing a program to take a backup of database. Want to zip database files. One way of doing that is, wait until the copy is done and then zip. But I would like to zip it while copying.

Would named pipes be a solution? Not sure how to use a named pipe in this situation...

Here is what I wrote, which does serial work, but want to make it to do those two in parallel:

#!/usr/bin/perl
use File::Copy;
use POSIX;
use Fcntl;
use IPC::Open2;
use File::Basename;

$sourcedir = qq(/home/oranvad/ap/source);
$targetdir = qq(/home/oranvad/ap/target);
$file      = qq(ijk);

my $cpstat = &copyFile($file);
print("\nfile=[$file]\tstatus=[$cpstat]");

sub copyFile() {
  my $fname = shift;
  my $cpcmd = qq(cp -p $sourcedir/$fname $targetdir/$fname);
  my $zpcmd = qq(zip $targetdir/$fname);
  print("\nCopying [$fname] with command [$cpcmd]...");

  sleep(5);
  system("$cpcmd");
  system("$zpcmd");
}
[download]

Comment on Zip files while copying Download Code

Replies are listed 'Best First'.
Re: Zip files while copying by ramprasad27 (Sexton) on Nov 16, 2011 at 02:56 UTC
two things.. 1)If source file is already present than just zip it by giving target file location this copies as well as zips 2) if source file is being created(data being written) than use fifo pipes and direct it to gzip -c which will create compressed files on the fly.. Now tell us what you are looking at	[reply]
Re: Zip files while copying by ambrus (Abbot) on Nov 16, 2011 at 08:53 UTC
I don't understand your problem. When you copy the file like `cp srcdir/file destdir/file`, could you just zip it instead like `zip destdir/file.zip srcdir/file` ? If not, could you please clarify your question, as in what are the statements that work but don't do what you want eg.?	[reply] [d/l] [select]
Re^2: Zip files while copying by aaron_baugher (Curate) on Nov 16, 2011 at 15:54 UTC
One concern might be that if you give zip a pathname, it'll have that in the archive, and will want to unzip it there. So in your example, it'll want to unzip it as srcdir/file. That may or may not be a problem for him. On the other hand, if he copies it and then zips it, it'll be in the archive and unzip as destdir/file. Really, although you can use zip on single files, it's really designed for making archives of directory trees. So he might be better served with something like gzip, that only compresses and doesn't try to know the filenames/paths. Aaron B. My Woefully Neglected Blog, where I occasionally mention Perl.	[reply]
Re: Zip files while copying by remiah (Hermit) on Nov 16, 2011 at 12:04 UTC
Archive::Zip was very good for me. "addTree" will do work for you.	[reply]
Re: Zip files while copying by calsaint (Initiate) on Nov 16, 2011 at 17:46 UTC
I got it going, thanks everyone `my $cpcmd = (dd if='$sourcedir/$fname' bs=512k\|gzip \| dd of='$targetdi +r/$fname.gz' bs=512k)` [download]	[reply] [d/l]
Re: Zip files while copying by calsaint (Initiate) on Nov 16, 2011 at 20:24 UTC
the source can not be zipped because as soon as the backup is done I would like to start the db, and not waste time zipping and unzipping @remiah:I did think about using that module, however as Aaron mentioned gzip worked out better to zip individual files. the one that I posted earlier is working. @ramprasad27: hoping that it answers your question based on what I have done, please see if there is a better way of achieving that	[reply]