Creating Tarballs in a perl scripts

mdw1982 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I've got a problem with a program I wrote a few years back - a backup program - that works fairly well and is maturing, however it does have a problem. One of the things this program does is create TGZ archives.

The problem I'm having, and I have yet to find a solution to this vexing issue, is that no matter what I do it will not grab .hidden files. I've tried altering the manner in which I issue the command to create the archive within the program; I've tried using the Tar module; I've tried turning on and off the BASH command to allow .hidden files to be scooped up, but all with no joy.

The command I'm using and have embedded in the program is this:

`tar -zcf $filename $DEST`;
[download]

It does a wonderful job of grabbing everything in the directory it's working in: all the directories its supposed to backup are kept in a database which it reads each time the program runs and loads the directory paths into an array. For the life of me I can't understand why it is not getting the hidden files. When I run the command from the command line all files are archived as expected including the .hidden files.

Can anyone shed some light on this vexing problem for me?

Comment on Creating Tarballs in a perl scripts Download Code

Replies are listed 'Best First'.
Re: Creating Tarballs in a perl scripts by morgon (Priest) on Jun 11, 2012 at 15:55 UTC
First of all: Do you really want the outout of tar? If not you should not use backticks but `system`. Then it all depends on what your variables acually contain as the string you use inside the backticks will be interpreted by a shell. If for example your $DEST variable contains "/tmp/" then it is clear why no hidden files make it to the tar-archive - the shell globs "" and will not pass any hidden files to tar. The shell used in backticks is usually /bin/sh (not bash - bash-settings are irrelevant here). If your $DEST does not contain shell meta-characters then I cannot see any reason why your command should not work, but that would not be a Perl-related problem. Finallly if you use system you can avoid a shell by calling it like `system("/bin/tar", "-cfz", $output_file, $directory)` [download]	[reply] [d/l] [select]
Re^2: Creating Tarballs in a perl scripts by Lotus1 (Vicar) on Jun 11, 2012 at 19:39 UTC
First of all: Do you really want the outout of tar? If not you should not use backticks but system. Perhaps the OP doesn't want to see the output on the display.	[reply]
Re^3: Creating Tarballs in a perl scripts by rovf (Priest) on Jun 12, 2012 at 08:36 UTC
Perhaps the OP doesn't want to see the output on the display. IMO it would be natural (and easier to understand) in this case to redirect to the bit bucket... -- Ronald Fischer <ynnor@mm.st>	[reply] [d/l]
Re^2: Creating Tarballs in a perl scripts by mdw1982 (Initiate) on Jun 12, 2012 at 16:54 UTC
First, thank you all for your replies; much appreciated as this has been bugging me for quite some time. Morgan, your solution was just what I needed, interestingly though when I tried using system() before I couldn't get it to work as I wanted. Implementing the way you exampled did the trick. At first it wasn't working correctly, but that led me to a glaring error in the code that I'd been missing. All is fixed now and working nicely. Thank you all for your assistance.	[reply]
Re^3: Creating Tarballs in a perl scripts by morgon (Priest) on Jun 12, 2012 at 17:48 UTC
Morgan, your solution was just what I needed Actually it's morgon (as in the Riddlemaster of Hed) and not Morgan (which is a fairy queen). But glad I could help.	[reply]
Re: Creating Tarballs in a perl scripts by zentara (Cardinal) on Jun 11, 2012 at 17:03 UTC
I just tested this and it grabs hidden files. `#!/usr/bin/perl use warnings; use strict; my $dir = '.'; system ('tar', '-zcvf', 'my_tarball.tgz', $dir);` [download] I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply] [d/l]
Re^2: Creating Tarballs in a perl scripts by stevieb (Canon) on Jun 11, 2012 at 17:29 UTC
Using zentara's code, I also confirm it working on Ubuntu 11.10, Ubuntu 12.04 and FreeBSD 9.0.	[reply]
Re: Creating Tarballs in a perl scripts by rovf (Priest) on Jun 11, 2012 at 15:36 UTC
Well, there is for sure a difference, whether you put this code into Perl, or into the shell. For example, depending on the value of $DEST, Perl might decide to pass on the command to the shell for evaluation, or call tar directly. In order to discuss this, we would need to know the interpolated string - something like my $cmd="tar -zcf $filename $DEST 2>&1"; my $cmd_output=`$cmd`; print "Command $cmd produced the output:\n$cmd_output\n"; [download] would do. -- Ronald Fischer <ynnor@mm.st>	[reply] [d/l]
Re: Creating Tarballs in a perl scripts by Illuminatus (Curate) on Jun 11, 2012 at 16:16 UTC
There's nothing magic about '.' files as far as tar is concerned. Without a more complete sample of what you're doing beyond just the tar command used: setting of variables used by command error-checking code whether you do anything with the tarfile after you create it ... it's hard to suggest what's wrong. About the only thing that would prevent tar from including files would be permissions. The single snippet you provide doesn't indicate that you're checking the output of the tar command, which would write things like "tar: dir/.please_include_me: Couldn't visit directory: Permission denied" to stderr if the user the script is running as didn't have permission to read the file. Without seeing the code, I would suggest adding the -v option to the command in your script and capturing all of stdout/stderr. It might tell you something fnord	[reply]
Re^2: Creating Tarballs in a perl scripts by aaron_baugher (Curate) on Jun 11, 2012 at 20:53 UTC
The way dotfiles are usually missed in a situation like this is when the command is given a wildcard: `tar -czf file.tgz ` That will tar up everything beneath the current directory, except* for any dotfiles in the current directory, because * in a shell doesn't match those (normally, though some shells might under some circumstances). The shell will interpolate the * to mean all non-dotfiles in the current directory, and tar will get them and anything under them (including any dotfiles in subdirecories). The usual way around that is to tar up the current directory, rather than the contents: `tar -czf /tmp/file.tgz .` [download] I don't know if this was the OP's problem, since I don't know what he had in `$DEST`, but it's a possibility. Aaron B. Available for small or large Perl jobs; see my home node.	[reply] [d/l] [select]