in reply to CGI upload efficiency

I looked into the hook feature, but as far as I understand it, that is only a means to monitor the upload process.

No, you don't understand it completely. The upload hook is a callback that you create. It is called repeatedly while your file is uploading. The $buffer argument (see the docs) is how you get at the uploaded data. You could, for instance, write that out to whatever file you choose.

A couple other things... With the way you are currently doing it, I'm pretty sure CGI.pm should just be renaming the temporary file. If the source and destination drives are the same that should be nearly instantaneous regardless of file size.† Using a different drive will force it to copy the data which is not what you want. Also, It's not documented and therefore not advisable, but you can set $CGITempFile::TMPDIRECTORY to choose a different directory for your temp files.

Here's a quick sample script so you can see what the upload hook does. Upload a smallish text file to see it work. The $data argument can be anything you want. (You might want to pass a filename or filehandle that you'll use within the hook to write the data to, for example.)

#!/usr/bin/perl use CGI; my $q = CGI->new(\&hook, 'some arbitrary data', 0); print $q->header(); form_page(); my $header_printed = 0; sub hook { print "Content-type: text/plain\n\n" unless $header_printed; $header_printed = 1; my ($filename, $buffer, $bytes_read, $data) = @_; print "Read $bytes_read bytes of $filename\n"; print "My \$data = $data\n"; print "Buffer length: " . length($buffer) . "\n"; print $buffer . "\n"; } sub form_page { print $q->start_html(); print $q->start_multipart_form(); print $q->filefield( -name=>'uploaded_file', -default=>'x', -size=>50, -maxlength=>80); print $q->submit; print $q->end_form; print $q->end_html; }

† I don't do Windows. Maybe I'm giving it too much credit for being sane here.

Update: Added example code.

-sauoq
"My two cents aren't worth a dime.";

Replies are listed 'Best First'.
Re^2: CGI upload efficiency
by Anonymous Monk on May 08, 2012 at 22:27 UTC

    Thanks for the example, I'll try it straight away.

    my $q = CGI->new(\&hook, 'some arbitrary data', 0);

    If I read this correctly, the '0' tells CGI not to write data to it's normal file and therefore the bits will only end up going wherever &hooks directs them to?

    Regarding the temp file location, I had trouble changing the location in code, but setting the TMPDIR env variable in the OS worked for what I needed.

    The behavior I saw was that even on the same drive CGI made a copy of the file and then unlinked the CGITemp one. On a 2GB file the delay was quite noticeable, so I used sysinternals procmon to verify that the data was infact copied, even when on the same disk. It could work differently on linux, no doubt. It really wouldn't be an issue either way if the files were a more reasonable size ;)

      If I read this correctly, the '0' tells CGI not to write data to it's normal file and therefore the bits will only end up going wherever &hooks directs them to?

      Yes. If you instantiate CGI.pm this way, it won't write the uploaded data to a temp file. Your only chance to do something with it is as you get it chunk by chunk in your callback routine (the hook function.)

      -sauoq
      "My two cents aren't worth a dime.";
Re^2: CGI upload efficiency
by Anonymous Monk on May 08, 2012 at 07:40 UTC

    Also, It's not documented and therefore not advisable, but you can set $CGITempFile::TMPDIRECTORY to choose a different directory for your temp files.

    IIRC it has officially documented for at least a year