BKoT has asked for the wisdom of the Perl Monks concerning the following question:

I come as a self-proclaimed n00b seeking wisdom concerning output threading. I've written a very basic routine to retrieve a list of files and push them to a web user via archive::zip. In the code below (please ignore some of the obvious newbie techniques) you'll see that the zip file is being written directly to STDOUT. (This keeps me from having large .zip files lying around that would have to be cleaned up later.) I'd like to go to the next step and give the user the ability to use download accelerators, but given the way the .zip file is being created, I'm not sure if it is even possible to open multiple output threads. Am I wrong? Any little push in the right direction would be greatly appreciated.
#!/usr/bin/perl -w use CGI ':standard'; use String::CRC32; use Archive::Zip qw(:ERROR_CODES :CONSTANTS); $myname = $0; $myname =~ s/\///g; $myname =~ s/usrlocalweb//; $myname =~ s/.cgi//; $zip = Archive::Zip->new(); print "Content-disposition: filename=$myname.zip\n"; print "Content-type: Application/zip\n\n"; open (LISTIT, "<$myname.txt"); while (<LISTIT>) { $_ =~ s/\n//g; push @filenames,$_; } $filecount = ($#filenames)+1; close (LISTIT); for ($loop=0; $loop < $filecount; $loop++) { my $member = $zip->addFile( $filenames[$loop] ); $member->desiredCompressionMethod( COMPRESSION_DEFLATED ); $member->desiredCompressionLevel( 9 ); } $output = $zip->writeToFileHandle( STDOUT ); exit $output;
Thanks, -BKoT-

Replies are listed 'Best First'.
Re: Threading http file transfers
by iburrell (Chaplain) on Dec 20, 2002 at 18:09 UTC
    There is no point in having multiple output threads serving a single HTTP request. There might be some benefit to having one thread handle any calculation (creating the zip files) and another sending the data back to the client.

    Download accelerators work by dividing the file into ranges and requesting each range in a separate connection. The easiest way to serve these requests is cache the zip file on disk. Create it on the first request and serve the proper ranges on later requests. It might even be possible to allow the web server to serve the zip file and only call your script when it doesn't exist.

    The other option is for each request to create the zip file, serve the range requested, and then delete the temporary file.