Macphisto has asked for the wisdom of the Perl Monks concerning the following question:

I've run through the archives looking for as much as I can regarding a CGI or mod_perl file upload progress bar, and seen the same thing over and over again. "We're simply not sure." Basically, what I'm looking for right now is to possibly outline the situation I am facing, where I am, and to possibly hear a couple well thought-out suggestions, possibly a curmudgeoney answer, possibly a funny answer, i'd like a cute answer and heck maybe even a plain old spiteful answer.

Basically the long and short of what I'm dealing with is that I heard the words, "I like the app so far, but some people complain that when they upload a large file they can't tell if anything's actually happening. Any chance you can come up with a Progress Bar?"( ::sfx - Jarring Chord as men in white coats stand outside my office door waiting for me to crack :: ). I'm looking for a perl way to do this. The people who are going to be using this interface are not the type of people who would be apt to installing a plug-in, and let's be honest, javascript just leaves a funny smell on our hands.

I've been using mod_perl for this and what I've noticed is as far as I can tell, within a multipart/form-data submission the contents of the file are actually transmitted once you hit the submit button. Hence, I can't get a request to be handled and start counting the temporary file before the file even finishes uploading. I simply don't get to touch the process before it's uploaded. This is where I'm stubbing my toe against the situation. Every answer I can come up with has to assume this fact isn't there.

I keep trying to think of a way I can beat that request to the punch. To have something waiting to grab that information first, gather the total size of the file being sent, then start counting as the temporary file fills up. This way I can return information in the form of an on-the-fly GD Image as the file is being uploaded. However, I'm not even sure if this is a perl issue so much as hack on Apache.

I'm pretty much out of steam right now. More will come to me if this gets any discussion.

Macphisto
Everyone has their demons...you just happen to be mine.

Replies are listed 'Best First'.
Re: Yet another progress bar.
by Kanji (Parson) on May 29, 2002 at 05:50 UTC

    I've only just started dabbling with mod_perl, but it looks like Apache::Request->new's UPLOAD_HOOK argument might be of use...

    UPLOAD_HOOK

    Sets up a callback to run whenever file upload data is read. This can be used to provide an upload progress meter during file uploads. Apache will automatically continue writing the original data to $upload->fh after the hook exits.

    However, I think the real trick will be "streaming" your progress meter back to the client as I've never had much luck in getting the browser to read anything back from the window or frame it's uploading to until after the upload is done.

    However, mod_perl certainly gives you a much finer control of interaction than CGIs, so you might have some joy there or with nph although personally I've had much success using multiple windows (the progress window being launched via JavaScript) or frames, and then employing some sort of communication between the upload and progress processes, with the window/frame for the latter refreshing often.

        --k.


[reply]
Re: Yet another progress bar.
by ajt (Prior) on May 29, 2002 at 07:47 UTC
    We had the exactly same problem with an application I use to work with. Customers didn't know what was going on, and felt things were taking to long to complete.

    We cheated and simply used Javascript to open another window, and show an animated gif. There was no progress bar shown, no kind of attempt to show percentage done at all, it was a pure con. However most people were fooled, and thought it was much nicer, and even that the processes were running quicker - which they were not.

    Useability Guru Jakob Nielsen has a lot to say about this kind of feedback at Use IT, and even though it's hard to do, it's not an unreasonable request.

    My humble virtual 2p

[reply]
Re: Yet another progress bar.
by Nomis52 (Friar) on May 29, 2002 at 06:24 UTC

    I've seen this done quite a bit using servlets and ASP, but never using Perl :(

    As far as I can tell here's the basic idea:

    Put a onSubmit javascript handler so that when the form is submitted, a new window is opened calling say progress.cgi

    The original page then goes ahead and submits the form, somehow you have to capture the total size of the file before it is uploaded and store it somewhere (perhaps in a database) so progess.cgi could access it.

    The progress window is then set to refresh every say 2 seconds, when it does it checks the temporary file size and grabs the total size out the database. Then you can make up an image or whatever your after.

    There is a demo (regrettably not in Perl but VBScript) here

    I have been meaning to look into something like this for a while but havn't had the time, so please keep us informed of your progress (no pun intended)

[reply]
Re: Yet another progress bar...addendum
by Macphisto (Hermit) on May 29, 2002 at 05:38 UTC
    The following approach just came to me while I was settling into bed, and since my memory is terrible, I thought it best to write it up quickly:

    If I could somehow, not process the information immediately after the submission button was pressed, I could then at least produce a graphic, or just throw out a count of bytes recieved. Something akin to - the user submits the information by striking the submit key and before I start pulling down that information, I create a popup window ( or just use the mainwindow for all I care ) and display total kB of data I've recieved by using the -s test on a predetermined temporary file. This would of course start at 0 since the file would be there, but it would have no data initially. The transfer of the file would begin and a meta-tag Refresh would allow the page to refresh every three seconds, thus recalling the progress_handler and again -s testing the temp file. The problem is how to delay that download so I can make the initial call to progress_handler and output output something - anykind of html or display first?



    Everyone has their demons...you just happen to be mine.
[reply]
Re: Yet another progress bar.
by valdez (Monsignor) on May 29, 2002 at 10:12 UTC

    You can try to use Server Push (see your CGI.pm man page and examples) to serve out a progress bar. Basically the solution can be a server push for each uploaded chunk of file.

    The solution should be similar for mod_perl, using the hook provided by Apache::Request, as stated by Kanji.

    Please note that i didn't try it and that man page says "Only Netscape Navigator supports server push. Internet Explorer browsers do not".

    Hope this helps.
    Bye, Valdez
[reply]
Re: Yet another progress bar ... progress report.
by Macphisto (Hermit) on May 29, 2002 at 15:42 UTC
    Okay, so I had to use a wee bit of javascript. I'm not happy about it. I'm seeking counseling for it. Do you know of a good doctor?

    Anywho, I got a minimal case working. I'm storing session information with a PostgreSQL database using a random 10 alphanumerics for session_id, so I decided to use the session_id as temporary filename and to rename the file after upload was completed. When a user hits "Upload" there is an OnClick setting that will open a new window( the javascript, it leaves a funny smell on my clothing! ) which due to its title( inventively: /upload/progress ) and some params tacked onto it, gives the progress handler all the information it needs. Every three seconds the window reloads, -s'ing the session_id which is a filename. Currently, I haven't played around enough to see if I can get the total size of the file before hand so I can produce a percentage or a keen little graph. This leads me to the conclusion that a progress bar is highly possible. Maybe my hack isn't the most inventive or pretty thing in the world. However, I'm far from being one of the better perl hackers on this board and I'm sure someone could come up with a more elegant way of doing this. Again, impossible is not a word that perl takes lightly.

    Everyone has their demons...you just happen to be.
[reply]

      Just suggesting a bit of useless glitz here, which could make it look smoother:

      I was initially going to suggest you try putting an initially empty IFRAME into your upload page, wherein the Javascript then loads the progress meter CGI OnSubmit(). But IFRAMEs are the devil. Even so the idea itself has merit..

      But let's take it to a whole new dimension. Place an initially empty graphic on your page (the transparent singlepixel GIF will do). Your Javascript shall then reload that image from the progress meter CGI's address, which now produces image/png rather then text/html, complete with a Refresh: header. This method should work for any but the very dumbest of browsers and will provide the feedback completely smoothly integrated into your upload page. Now you have the mirrors.

      It's time to add the smoke: have the Javascript replace another, static image of for example a clock with an animated version. Your users will never even know what hit them.

      David Copperfield made the Statue of Liberty disappear, why couldn't you make a progress meter? :-)

      Update: Darn. As Macphisto pointed out, the page with the upload form will be frozen while the upload happens, with no chance to update any images in it. In that case, I guess the only way to resolve this is to use frames in some fashion.

      Actually, it should be possible - the FORM tag has a TARGET attribute just like the A anchor tag does, if memory serves. In that case one could launch the upload into a different frame and set up abovementioned smoke and mirrors. It will require funny Javascript posing though..

      Guess a progress popup really is the only actually workable solution. :-(

      ____________
      Makeshifts last the longest.
[reply]
Re: Yet another progress bar.
by Anonymous Monk on May 29, 2002 at 12:22 UTC
    This is impossible, due to the fact that HTTP is a Request-Response protocol. That is, the server has to wait for the entire request (including the file) before it gets to send anything back.

    The best you can do is some kind of javascript popup that says something like "Uploading, please wait."

    Christoffer Hammarström
    kreigerATteliaDOTcom
[reply]
      You are right of course :)
[reply]

Back to Seekers of Perl Wisdom