isync has asked for the wisdom of the Perl Monks concerning the following question:

Am I overseeing something? I'd like to create (possibly) large zip-files in a web-app, and from my reading of Archive::Zip, it has no means to write to a (temp) file on disk - right?
If so, any options besides system("zip ...")?

Update:
My question might be misunderstood. I'm not asking about outputting to file, but about when the file is being assembled, when the zip grows, members being added, etc. - all that happens in memory, not on disk. So a, let's say, 150MB zip will take up 150MB RAM until it is written to disk, right?
(and let's put aside that this process may take some time and a web-app would block until the zip is done... - all that better be laid off into an async process...)

Update 2:
Archive::Zip::SimpleZip can do it.
It's a wrapper around IO::Compress::Zip and offers an interface similar to Archive::Zip, only it is able to "bind" it to a real/on-disk file on OO construction, and will then write added members to disk as they arrive.
That said/found, for my webapp scenario, it's probably less ideal to build a file to disk, having disk trashing in favour of a small memory footprint, and then send_file that to the client. A better solution is probalby to use Archive::Zip::SimpleZip's ability to output a streamed zip to STDOUT, so the file is transmitted to the client while it grows, memory footprint is low, and disk IO is only read()s for adding members.

Update 3:
Archive::Zip and Archive::Zip::SimpleZip are both able to build a zip file while keeping memory footprint low. (See pmqs's answer for the why and how)
There are small differences in behaviour though, for example A::Z::SimpleZip might be more appropriate when your zip member contents are not coming from file-system.

Replies are listed 'Best First'.
Re: Archive::Zip can only create zip-files in-memory, right?
by GrandFather (Saint) on Jul 26, 2014 at 02:39 UTC

    From the Overview section of the documentation:

    Archive::Zip::Archive objects are what you ordinarily deal with. These maintain the structure of a zip file, without necessarily holding data. When a zip is read from a disk file, the (possibly compressed) data still lives in the file, not in memory. Archive members hold information about the individual members, but not (usually) the actual member data. When the zip is written to a (different) file, the member data is compressed or copied as needed. It is possible to make archive members whose data is held in a string in memory, but this is not done when a zip file is read. Directory members don't have any data.

    which implies to me that the .zip is not built in memory as items are added so no problem eh?

    Perl is the programming world's equivalent of English
[reply]
      Right, but that applies to having a source zip to start from, changes/appended zip members then reside in memory, until you write back out to a file. Building a file from scratch means all members remain in memory until flushing to file.
[reply]
Re: Archive::Zip can only create zip-files in-memory, right?
by Your Mother (Archbishop) on Jul 25, 2014 at 20:46 UTC

    HURR? From the Pod: $zip->writeToFileNamed('someZip.zip').

[reply]
[d/l]
      I know.
      But from how it is presented in POD, it's just a "output to file" method: it's always last after constructor, adding members, etc. And from looking into the source, it's just that, not a getter/setter method. So it seems: Archive::Zip = only in-memory.
[reply]
Re: Archive::Zip can only create zip-files in-memory, right?
by isync (Hermit) on Jul 26, 2014 at 12:31 UTC
    I thought a quick post on PM would bring clarity from someone who knows for sure in a short thread. (Saving me time of digging through the module's source.)
    Now, with a number of nebulous answers, I begin to think, this is becoming more of a PM Medidation...

    From what I've read since the original post, Archive::Zip can only build zips in-memory. *wrong*, read on (it's actually quite smart)
    IO::Compress::Zip as an alternative has finer grained IO control, but offers less control over zip member (re-)naming.
[reply]

      Archive::Zip will "remember" all the actions you request done for the zip file, then do all of them in sequence when you do 

      $zip->writeToFileNamed('someZip.zip')

      [download]

      So if you are creating a zip based on real files from the filesystem, the contents of those files are not stored in memory. It just remembers the names of the files in memory.

      When you do make the writeToFileNamed call it reads the input files and writes the compressed data directly to the output filehandle for the zip file.

      Regarding Archive::Zip::SimpleZip - it does write to the zip file as soon as possible.

      One thing to be aware of though - the zip container format that gets used by 99% of all zip archives out in the wild needs to be written to a seekable filehandle when you are using Archive::Zip::SimpleZip. STDOUT is not seekable.

      Luckily, the Zip container spec does support a proper streaming format that can be used when writing to STDOUT. The issue then is to ensure that the client supports that type of Zip container. Most do these days.

      If you use Archive::Zip::SimpleZip with STDOUT, it will automatically create a streamed zip container. Whether that is acceptable depends on your clients.

      If you think there is an issue with member naming in Archive::Zip::SimpleZip I'd like to hear what is missing.

[reply]
[d/l]
        Exhaustive answer. Thanks! +1
        And reveals that I haven't dug around in Archive::Zip's source enough...

        (Nothing's missing in SimpleZip, as it adds the sugar needed to add a member under a different name - what I think, from reading the POD, is not possible with IO::Compress::Zip alone.)
        Case closed.
[reply]

Back to Seekers of Perl Wisdom