This problem can be demonstrated much more simply with this code:

#!/usr/bin/perl -wT

use strict;
use CGI ':standard';

die "Could not fork()\n" unless defined (my $kidpid = fork);
if ($kidpid) {
    print header;
    print "done\n";
    print STDERR "Parent exited\n";
    exit;

}
else
{
    sleep 20;
    print STDERR "Child exited \n";
}
[download]

If you run this CGI at a command prompt, it does what you would expect - the parent exits and you get the command prompt again. 20s later, the child exits and you see the output broadcast in the terminal.

Now run it as a CGI, and although you see "done" printed straight away in the browser, the page still appears to be loading until the 20s is up (the server appears to still be holding the connection open). A clue I found at http://coding.derkeiler.com/Archive/Perl/perl.beginners/2003-11/1136.html implies this could be to do with Apache. I found this issue running with mod_cgid in Apache 2.2.9 on Linux kernel 2.6.27.

My application is AJAX and is experiencing this problem; AJAX cannot solve it directly. My AJAX call cannot determine if the task started successfully, because it does not get the result until it ends. I am going to see if I can work around it by making the call and ignoring the response; hopefully the FF JS engine is multithreaded enough to let me do that.

I would love to hear from anyone who has managed to make Apache let perl fork properly in this scenario. Having said all that, if you read http://www.xav.com/perl/lib/Pod/perlfork.html, it says "the parent and every pseudo-child created by it that is also a pseudo-parent will only exit after their pseudo-children have exited.", so I'm not sure the command prompt example should work as it does.