in reply to Creating Zip Archives on the fly

See thread keeping connection alive while spending time building a zip file where a similar problem with generating large zip files on the fly was discussed.

Having the zip files already available is the best way to go with this. If that isn't an option, and you can't use one of the other suggestions in that thread, you will have to stream the zip file as it is created to the client.

That means two things:

  1. You need to use HTTP chunked transfer encoding to send the content to the device as you write it. How you get that working will depend on what web setver you are using.
  2. The zip implementation you use needs to support streaming output.

I din't think Archive::Zip supports streaming output. Both the command line zip and IO::Compress::Zip can stream a zip file as it creates it.

Below is a proof of concept code I posted in the other thread that streamed a zip file and chunked it at the same time using IO::Compress::Zip. 

use IO::Compress::Zip qw(:all) ;

select STDOUT;
$| = 1;

my $OUT = \*STDOUT;               

print <<EOM;
Status: 200 OK
Content-Type: application/zip
Transfer-Encoding: chunked

EOM

my @files = qw(/tmp/file1 /tmp/file2) ;
zip [@files] => '-',
        FilterEnvelope => sub         
            { 
                # Chunk the output
                my $length = length($_);
                $_ = sprintf("%x", $length) . "\r\n" . $_ . "\r\n";
                $_ .= "\r\n" unless $length;
                1;
            }
        ;

[download]

One thing missing from your original requirements is the ability to rename the zip file members as you create the zip file. That is a feature that is under development now for IO::Compress::Zip (I'm the author of the module). I can get you an early rlease if you want.

Replies are listed 'Best First'.
Re^2: Creating Zip Archives on the fly
by fulmar2 (Initiate) on Oct 05, 2011 at 17:04 UTC
    THIS, is exactly what I was looking for! Thank you. I might be able to "let go" of the renaming (not as critical as the streaming for our application). Nevertheless, I may eventually be interested in the next release of IO::Compress::Zip. Thanks again. Also, I agree about having the zip file already available... I considered that, but it's a pretty dynamic environment, so the best option will really be to build them on the fly.
[reply]

      Just a quick followup on this thread. The latest version of IO::Compress::Zip now has the ability to rename zip members on the fly using the FilterName option.

      Here is the proof-of-concept from before updated to include FilterName 

      use IO::Compress::Zip qw(:all) ;

select STDOUT;
$| = 1;

my $OUT = \*STDOUT;               

print <<EOM;
Status: 200 OK
Content-Type: application/zip
Transfer-Encoding: chunked

EOM

my @files = qw(/tmp/file1 /tmp/file2) ;
zip [@files] => '-',
        FilterName => sub
            {
               if ($DLWOSC eq "checked"){
                 $_=(split('/',$_))[-1];
                 s/\;/\_/g;
                }
            }, 
        FilterEnvelope => sub         
            { 
                # Chunk the output
                my $length = length($_);
                $_ = sprintf("%x", $length) . "\r\n" . $_ . "\r\n";
                $_ .= "\r\n" unless $length;
                1;
            }
        ;

      [download]
[reply]
[d/l]
Re^2: Creating Zip Archives on the fly
by Anonymous Monk on Sep 12, 2013 at 17:12 UTC
    For anyone looking at this comment FilterEnvelope is now FilterContainer
[reply]

In Section Seekers of Perl Wisdom