Here is how I did it...I created another (unique) HTML file with meta tags which will cause the page to refresh itself every 15 seconds and with no cache saved. Once that placeholder HTML file is created, redirect the user to that placeholder HTML file and kick off a grandchild worker to update the HTML file dynamically. Once the grandchild is done updating the HTML file then we clear out the HTML one final time with just the final HTML results you want shown.

CGI - Parent worker: /srv/www/cgi-bin/parent.pl

• obtain CGI params like normal
• create directory and html file and give proper permissions for both
• printf HTMLFILE "<meta http-equiv=\"refresh\" content=\"15\">"
• printf HTMLFILE "<META HTTP-EQUIV=\"Pragma\" CONTENT=\"no-cache\">"
• printf HTMLFILE "please wait message or an animated gif"
• print "Location: PlaceHolderURL\n\n"
• fork off a child worker (passing in the html filename)
• wait 3 seconds to allow the child to start up
• exit

CGI - Child worker:

• close stdout (close STDOUT;) and stderr (close STDERR;) so browser doesn't wait on child
• exec to a grandchild which will do the work, passing in the html filename

CGI - Grandchild worker: /srv/www/cgi-bin/grandchild.pl

• get html file as an argument
• append html file with any progress you want to show and do long-running processing
• once finished processing:
  • -optional: close and reopen html file without appending to clear it
  • -print out what you want shown (the html results)