tar_mv: move tar data as it's being extracted

My friend asked me today how to change the destination directory of extracted tar data. At first I suggested

tar xvf blah.tar > /some/dir

but that was plain old silly. Then I peeked at the manual and couldn't find anything; the closest seemed to be -C flag, but I couldn't quite get it to do what we wanted.

Next I tried hacking the tar header, but that failed too.

Suggesting moving the tar file to some directory, extracting , and moving stuff back up to the new directory wasn't a good solution for him either.

When reading the manual, I noticed the -O flag, which extracts a tarfile to stdout. Mingled among the extracted data are the filenames that are being extracted. Voila, tar_mv.pl was born.

I doubt this can handle binary files very well (haven't tested yet) but it works well on multi-level tarfiles that are text-only.

#!/usr/bin/perl

BEGIN {
  sub usage {
    print "usage: tar xvfO tarfile.tar |& $0 ", 
          "/path/to/newdir\n";
    exit 1;
  }

  if ( $^O ne 'linux' ) {
    print "tar on $^O doesn't support O flag\n\n";
    usage();
  }

  -p STDIN or usage();    # make sure we're in a pipe
  @ARGV == 1 or usage();
  $main::newdir = shift(@ARGV);
  -d $main::newdir or mkdir $main::newdir;
}

use strict;
use warnings;

my $fh = ();
my $orig_dir = ();

while (<>) {
  chomp;

  # grab original root dirname from tar header
  if ( not defined $orig_dir ) {
    #($orig_dir) = m {^([^/]*)/$} ;
    # Remove /$ to allow handling sun- & alpha-made tarballs
    ($orig_dir) = m {^([^/]*)} ;
  }

  else { 
    # change root dirname in fileheader to new dirname
    if ( s{^$orig_dir/([^/]*?)}{$main::newdir/$1}x ) {
      $fh = get_newfh($_);
      next unless defined $fh; # Justin Case
    }

    # print extracted data
    else {
      print $fh $_, $/;
    }
  }
}

sub get_newfh
{
  my ($file) = shift(@_);

  # silly "open or do {...}" to handle subdirs in tar file
  open OUT, ">$file" or do { if ( $! =~ /is a directory/i) {
                               mkdir $file unless -d $file
                             } else {
                               die "$file: $!\n";
                             }
                           };
  \*OUT;
}
[download]

Suggestions/glaring bugs?

UPDATE

fixed regex that grabs $orig_dir; tarballs created on alphas don't have a trailing '/' on first line of "tar xvfO alpha.tar |& head -2"
base tar that ships with sun and alpha doesn't support O (extract to STDOUT) flag, so added die test in BEGIN
see discussion below about symlinks, permissions/ownerships, etc. getting, uh, lost.

blyman
setenv EXINIT 'set noai ts=2'

Comment on tar_mv: move tar data as it's being extracted Select or Download Code

Replies are listed 'Best First'.
Re: tar_mv: move tar data as it's being extracted by mojotoad (Monsignor) on Jul 26, 2002 at 22:44 UTC
The traditional approach...viva unix... No tarball other than in the streams... `$ (cd src_dir; tar cvf - .) \| (cd target_dir; tar xvf -)` [download] or never change your current directory, but wish to redirect from an existing tarball: `$ (cd tarfile_dir; tar xvf - tarball.tar) \| (cd tgt_dir; tar xvf -)` [download] Update:As mem points out below, the second example above is sort of meaningless -- no need for the pipe...this is what I meant: `$ (cd tgt_dir; tar xvf - src_dir/tarball.tar)` [download] In the cases where the tarball already exists, this presumes that the directory paths are relative vs. absolute. (as is pointed out below, you can use `-C` to redirect absolute paths to relative) Matt update: removed redundant first example, functionally nothing more than: `$ cd tgt_dir; tar xvf src_dir/tarball.tar` [download] Because I was thinking of: `$ gunzip -c tarball.tar.gz \| (cd tgtdir; tar xvf -)` [download]	[reply] [d/l] [select]
Re: Re: tar_mv: move tar data as it's being extracted by mem (Acolyte) on Jul 30, 2002 at 07:11 UTC
`$ (cd src_dir; tar cvf - .) \| (cd target_dir; tar xvf -)` Actually that's not what the original monk wants to do. Brother blyman wants to extract files inside a tar archive to a different location than the one they use inside the archive. For example, the archive contains `/usr/bin/perl` and he wants to be able to extract it to `/usr/local/bin/perl`. Your second example: `$ (cd tarfile_dir; tar xvf - tarball.tar) \| (cd tgt_dir; tar xvf -)` Doesn't do much, because it extracts `tarball.tar` from a tar archive coming via STDIN, and the second tar does nothing. Besides the problem already pointed out by Brothen blyman, I think there's a second one: permissions for directories are not preserved. If want to extract `/usr/bin/perl` to `/usr/local/bin/perl`, you'd like to preserve the permissions of at least `bin`. Inside the tarball there could be files as well as other little beasts, such as directories, fifos and devices. I was trying to come up with a solution that doesn't involve a temporary storage location but everything I've come up with has drawbacks (performance being the most common one). Looking at `Archive::Tar`, you can get a list of the files in the tar archive and then request the data for each of them. In principle you'll be doing something like: `my @files = $archive->get_files; foreach my $file (@files) { my $dst = compute_new($orig, $replacement, $file); save_data($file, $dst); }` [download] The problem is of course what I have pointed out, namely getting the permissions right. `save_data` here of course handles permissions and all the little details. What I wanted to make clear is that this requires you to read the tarfile twice: once for the list of files and a second one for the actual data. A forward iterator kind of interface would be helpful here. You would just get the name of the current file and request its data if needed or just skip to the next file. That would allow for a solution that works with pipes.	[reply] [d/l] [select]
Re: Re: Re: tar_mv: move tar data as it's being extracted by belden (Friar) on Jul 31, 2002 at 20:52 UTC
Actually that's not what the original monk wants to do. Brother blyman wants to extract files inside a tar archive to a different location than the one they use inside the archive. Correct - I wasn't able to explain it so succinctly, though, and didn't provide any examples of what I meant. Thanks for reading my code (and mind!) to figure out what problem I was trying to solve :) Besides the problem already pointed out by Brother blyman, I think there's a second one: permissions (and ownership) for directories (and files) are not preserved. Yep, I did some monkeying around and realized that my program wasn't getting that info, so couldn't possibly do anything with it. I believe that the original problem can be solved with a one-pass program because the intent is to extract all files to a different root directory; a future enhancement would be to extract only selected files to some arbitrary directory. I ++'ed you for a great reply; it's gotten me thinking in new directions, so thanks. blyman setenv EXINIT 'set noai ts=2'	[reply]
Re: tar_mv: move tar data as it's being extracted by Anonymous Monk on Jul 29, 2002 at 06:37 UTC
tar xvf blah.tar -C /path/to/where/you/want/it	[reply]