Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello my steemed code-friends!

 
Merry Christimas to all of you! (not sure what this date really means nowadays, but, anyway, cheers for everybody!)

 
I canīt find anywhere on the net - and the regular books donīt mention it - how to put a UPLOAD progress bar to show the visitor the progress of the file he is uploading. Iīm setting up a kind of marketplace where the vendors will be able to upload his catalogs. In the script thatīs triggered by the submit, I use CGI.pm's upload method to load the data in a handle, like tradition says, and also do a lot of processing in this .txt file to check it for errors before importing it to my database.

I was thinking what kind of approach is possible and the main problems I hit are:

1) I donīt understand how can I send the user any info about the upload in the meantime the scipt (upload.cgi) runs. It has something to do with returning a "no content" header to the browser? I mean, because now the script uploads the file, checks it and, at the end, presents another page showing the results of the analysis. How can I make my script multi-task? I thought that, for example, while the upload isnīt finished, Perl is stuck in the line where I call the ->upload method. Isnīt it? Can it do other things meanwile? It has to do with setting buffering off as I suspect??

2) Also, how can I access this info about the amount of the file that has already been uploaded by CGI.pm? Is it available through any variable?

3) Ok, if I manage my script to be multi-task, and talk with the userīs browser while it uploads the file, telling it not to move untill the processing finishes and finally I can show the analyss page, how do I manage to have the already-loaded page to suddenly go alive by itself showing the status? Through javascript? Java? Flash? How to make the progress bar a REAL progress bar, I mean, showing the real progress and not running the risk of showing 70% when the upload is in fact already done?

Hey, can anyone therehelp me out on this? Iīm completely lost. Iīm becoming very confortable with Perl processing that uses traditional CGI calls and resources, but this one is far away from my reach now. My only hope is thy wisdom, perlmonks!

Thanks a lot, folks!

André
Greetings from Brazil!

Replies are listed 'Best First'.
Re: Upload Progress Bar
by Aristotle (Chancellor) on Dec 25, 2004 at 06:18 UTC

    There is no good way to do this. It really is the responsibility of the browser to show such a progress bar, but none of them do.

    The common way to hack around this is to put some Javascript on the submit button that opens a popup before submitting the form. This popup then periodically refreshes from some script on the server which monitors the progress of the upload. It's a really fugly kludge but about the only way to do something about this.

    Makeshifts last the longest.

      PHP File Upload progress without modifying the PHP interpreter or recurring to Perl or ASP just as described in the previous comment... Not so ugly. ;-) http://tinyurl.com/66fkq
Re: Upload Progress Bar
by redhotpenguin (Deacon) on Dec 25, 2004 at 19:22 UTC
    I had to solve this exact problem about 6 months ago for an application that parsed uploaded files real time and fed back progress indicators to the user. There is a great description of how to do a search in progress out there by merlyn, but this situation is a bit different in that the main application window is handling the upload, and cannot be refreshed.

    I did this in a mod_perl2 handler using Apache::Request, but you should be able to accomplish the same with cgi scripts. Upon the user clicking 'Upload', I used javascript to open a small status window which refreshed waiting for the presense of upload status data. Meanwhile the main window begins the upload process. An UPLOAD_HOOK (see a::r docs) subroutine measured the amount of bytes received from the client and wrote it to a status file which the monitor read.

    Now, this worked very well for displaying the amount of bytes received from the user. As for determining the percentage of file uploaded, a::r provides the following useful method:

    my $size = $upload->size;

    However I did not find the size returned to be reliable when referenced from within the UPLOAD_HOOK subroutine. I suspect is determined once the file is fully uploaded. So I wasn't able to give the user a realistic estimation of when the file would be done uploading.

    But I was able to parse the contents real-time, display the results to the user, and terminate the upload if the errors in the file exceeded a certain threshold, which got good feedback from the users. I got some flak from reviewers of the technique, but it worked very well. I was able to pass back to the user accurate information about where in the file errors were, which saved them a great deal of time.

    HTH

      Hey Red, Thanks a lot, thatīs exactly what I was looking for. Iīve visited CPAN and in fact I think it will do. But, tell me, I can run this module even without mod_perl? My host's server is Apache.

      The one thing that still remains a mystery to me is how to manage, and by how many scripts, the signals and the processing of the file. I still donīt know if I can have the same script that processes the file upload and processing, to check $upload->size, tell the status bar (in wich page?) the new status or to get lost as the upload ends.

      Other thing is the javascript. Iīm not very skilled in javascript. Can you send me your code, or some sample of it? I donīt have a clue of how to do it...

      Thank you very much!

      André

        For information on javascript check here.

        I had the upload handler write the byte count to a session file, and the monitor process which was opened through the javascript on the "Upload" button read from the same session.

        CGI.pm also has an upload_hook subroutine which you can use to do this - you don't explicitly need a::r. You might also want to take a look at Apache::UploadMeter , I was using mod_perl2 so I had to roll my own.

Re: Upload Progress Bar
by zentara (Cardinal) on Dec 25, 2004 at 11:14 UTC
    There is a project called Mega-Uploader for PhP, I hacked a version of it in Perl. You can try it here. One is a pure javascript animation( which is probably better since it don't waste bandwidth) and the other is a real feedback system like Mega-Uploader. The javascript version requires that you have javascript enabled for changing images, and show all images is enabled in the browser.

    I'm not really a human, but I play one on earth. flash japh
Re: Upload Progress Bar
by superfrink (Curate) on Dec 27, 2004 at 09:09 UTC
    I used a Java tool called RadUpload ( http://www.radinks.com/upload/docs.php ) last year sometime. It requires some modern-ish java be installed (ie something more current than the win2k dev machine I had access to at work).

    You configure the Java applet to draw a box on the webpage and you set a parameter with a URL to post HTTP form data to. The user can then drag and drop files or directories onto that box on the webpage (assuming windows, osx, or a window manager that does that sort of thing).

    Yes I said directories. The java program sends the entire path info with each file. Then the CGI program can figure out where to copy the POST file data.

    Anyway the whole point is that this Java tool draws a progress bar on the client side based on how much data has been sent. Logically this job belongs on the client and not via periodic requests to the server. I have no idea why browsers don't just incorporate progress meters on file uploads.
Re: Upload Progress Bar
by TedPride (Priest) on Dec 27, 2004 at 07:26 UTC
    I tried doing this without a popup, with minimal success. IE just won't interpret the page properly in real time, so my progress bar ends up updating in chunks, or with strange effects:

    http://www.home-school.com/cgi-local/update.pl

    Oddly enough, the same page worked just the way I intended in Netscape 3.x, updating the bar by 1% every 3 seconds. Can anyone explain why IE isn't doing that? IE runs the update code, but most of the images are replaced with nothing until the page finishes loading, instead of the new image.

Re: Upload Progress Bar
by JSchmitz (Canon) on Dec 27, 2004 at 13:47 UTC
    Have you thought of using Tk? Something like this would work - then you could bind the action to the upload somehow?
    #!/usr/bin/perl use Tk; use Tk::ProgressBar; my $mw = MainWindow->new(-title => 'Processing'); $Progress = $mw->ProgressBar( -width => 30, -from => 0, -to => 100, -blocks => 50, -colors => [0, 'green', 50, 'yellow' ,80, 'red'], -variable => \$percent_done )->pack(-fill => 'x'); $mw->Button(-text => 'Go!', -command=> sub { for ($i =0; $i < 1000; $i++) { $percent_done = $i/10; print "$i\n"; $mw->update; } })->pack(-side => 'bottom'); Mainloop;
      Hey everybody!

      Thanks for the effort in helping me out on this! I have found many javascripts that output the progress bar, but the main problem remains: itīs not an actual progress bar. And the upload speed depends on too many elements for me to set up the speed of the blind journey to the 100%.

      The solution, as you already pointed, might be to use a upload hook, from CGI.pm or from Apache::Request. But the thing is that I just canīt understand the syntax:
      $q = CGI->new(\&hook);
      sub hook {
      my ($filename, $buffer, $bytes_read, $data) = @_;
      print "Read $bytes_read bytes of
      $filename\n";
      }
      (CGI.pm's upload hook.) Can anyone help me out on this? Wich variables should I replace here? Shall I put this on the same script where I do the upload routine?
      my $fh = $q->upload( "tabela" );
      Thanks! André

Re: Upload Progress Bar
by mkirank (Chaplain) on Dec 27, 2004 at 09:55 UTC
    Merry Christmas to all
    I was just going through a blog
    Here .Google uses XmlHttp in Javascript to complete the word
    I am not sure of the security considerations on this but I guess we can xmlhttp and come up with some sort of file upload progress moniter .
Re: Upload Progress Bar
by Qiang (Friar) on Dec 28, 2004 at 07:28 UTC
    superfrink gave a similar link but i am surprise that he didn't mention the perl version from the same project.

    its sourceforge project in java/perl/php. I just tried the demo and it looks pretty nice. here is the link.

    http://www.raditha.com/megaupload/perl.php

Re: Upload Progress Bar
by Anonymous Monk on Dec 29, 2004 at 15:58 UTC
    A useful feature for cases like this is the multipart/replaced content type. It allows the server to keep replacing its returned data over time, as long as the connection is open. Combined with the Javascript XMLHttpRequest it's a very powerful feature, as it allows to set a callback for when the server pushes more data. Of course, like with most of cool Web features, it does not work in IE (works in Mozilla/Firefox though). Thus, it's a pretty useless suggestion for this case. However, for those of you that, say, have Firefox as a standard corporate browser, that's another nifty tool for writing those crazy Web apps. Cheers!