arc_of_descent has asked for the wisdom of the Perl Monks concerning the following question:


Hi Monks,

I'm using ActiveState's ActivePerl 5.6
on a windows XP box.
My configuration is a Pentium III (1 Ghz)
and 128 MB RAM.
When i try to use Archive::Tar to tar a directory tree
about 30 MB in size, i always get a virtual memory problem
I can hope this is not a windows problem, but an inherent
problem with Archive::Tar trying to consume all the memory
for its operation.
Following is the program flow
I'm using File::Find to generate a list of files
below a particular tree, and then passing that list
to Archive::Tar... to.. tar :-)
Any suggestions or any other methods, modules?

Thanx a lot
--
arc_of_descent

Replies are listed 'Best First'.
Re: Archive::Tar Memory Consumption
by PodMaster (Abbot) on Aug 22, 2002 at 07:23 UTC
    If you've checked Archive::Tar, you'll see that the last version of Archive::Tar to be released is .22.

    The last available version that works on windows is .071.

    As the module's documentation on cpan states, it was only in

    Version 0.20 Added class methods for creation, extraction and listing of tar files. No longer maintain a complete copy of the tar file in memory. Removed the data() method.
    I've tried compiling .20 among all the lastest versions of this module, but none of them work, and it appears as though the module isn't being maintained anymore.

    I suggest you try another module, like Archive::Zip (i don't know if it's plagued by the same problem), or just forget working with a module altogether (just work with the tar utility)

    ____________________________________________________
    ** The Third rule of perl club is a statement of fact: pod is sexy.

      I've tried compiling .20 among all the lastest versions of this module, but none of them work, and it appears as though the module isn't being maintained anymore.

      What is failing? Archive::Tar is a Pure Perl module, AFAIK. It needs no compilation. It just should work.

      If there's anything wrong with it, it must be platform specific quirks. No it doesn't look like it's a missing binmode(), there are plenty of those.

      I'll try and look into it myself, later today.

        ...actually, I think that Archive::Tar is a Pure Perl module, but it uses (though does not depend upon) another module for gzip compression which is binary and requires compilation.

        ...All the world looks like -well- all the world, when your hammer is Perl.
        ---v

        OK, I've had time to look at it. I found the actual culprit, but at the same time, I found some more errors that affect other platforms as well.

        The reason why it failed, is this snippet, at lines 237-239:

        if ($^O eq "MSWin32") { $fh = $_[0]; }
        where later on, this is done with it:
        $fh = Compress::Zlib::gzdopen_ ($fh, $mode, 0)
        gzdopen_ is a low level function in Compress::Zlib. Well, guess what: this function expect a fileno, not a handle. So this fails. This does _drat() (it should either goto &_drat; or return _drat();, but not just call _drat()). $! is not set, so $error isn't set to a true value, and that's why it fails in such an ungraceful way. But I digress.

        The solution is simple: set

        if ($^O eq "MSWin32") { $fh = fileno($_[0]); }
        instead. And now all tests succeed.

        Here's the whole diff (for Archive-Tar-0.22.tar.gz):

        --- Tar.pm Fri Apr 28 00:50:16 2000 +++ blib/lib/Archive/Tar.pm Sat Aug 24 15:06:04 2002 @@ -79,7 +79,7 @@ my $error; sub _drat { - $error = $! . ''; + $error = $! . '' || 'unknown error'; return; } @@ -235,7 +235,7 @@ or goto &_drat; if ($^O eq "MSWin32") { - $fh = $_[0]; + $fh = fileno($_[0]); } else { $fh = fcntl ($_[0], F_DUPFD, 0) @@ -248,7 +248,7 @@ # $fh = Compress::Zlib::gzopen ($_[0], $mode) # or &_drat; $fh = Compress::Zlib::gzdopen_ ($fh, $mode, 0) - or &_drat; + or goto &_drat; } else { $flags = fcntl ($_[0], F_GETFL, 0) & (O_RDONLY | O_WRONLY | O_RDWR +);

        I'll try to warn the maintainer.


      HI,

      Thanx for the info
      I should have RTFM :-)
      Do u mean the tar utility that comes with cygwin?
      I was thinking abt downloading the whole stuff
      If it works, then maybe i will
      --
      arc_of_descent

        tar for cygwin works fine in windows. I have a batch script I made to back up a win2k site. Just put cygwin1.dll in the same directory as tar.exe and bzip2.exe

        This works:

        system('data\tools\tar.exe cvf wwwroot.tar wwwroot'); $n = &getnowstr; print 'Made tar archive. Compressing.'; system('ren wwwroot.tar wwwroot.' . $n . '.tar'); system('data\tools\bzip2.exe -4 wwwroot.' . $n . '.tar'); system('move wwwroot.' . $n . '.tar.bz2 baks'); print 'Done. ' . chr(13) . chr(10); sub getnowstr { ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time); $year += 1900; $mon++; $nowstr = $year . sprintf('-%02d%02d-%02d.%02d.%02d',$mon,$mday,$hour,$min,$sec); }
        This is called from within a .bat file with a line that says just "utils\makebackup.pl".