Forking not completing

fightliteracy has asked for the wisdom of the Perl Monks concerning the following question:

As is known, maintaining a large email list requires email to be sent to many addresses. A convenient solution should be to fork off the process so the user doesn't have to wait at a browser window for completion. I have tried this method of forking and yet the forking stops invariably before it goes through the entire list, at random spots in the list. Usually, it gets through about 140 to 200 emails before giving up. Is this a specific server problem or a general issue. Code below, any comments would be helpful.

unless ($pid = fork) {
  unless (fork) {
    close (STDOUT);
    foreach (@emaillist) {
      if (check_email($_)) { 
        open(MAIL,"|$mailprog -t") || print "Can't open $!";
        print MAIL "To: $_\n";
        print MAIL "Reply-to: $adminemail\n";
        print MAIL "From: $adminemail\n";
        print MAIL "Subject: $subject\n";
        print MAIL "\n$body\n";
        close(MAIL);
      }
    }    
    &process_complete;                    
    exit(0);
  }
  print "Content-type: text/html \n\n";
  print "Sending emails in the background. close your
         browser";
  exit (0);
}
waitpid($pid,0);
[download]

Comment on Forking not completing Download Code

Replies are listed 'Best First'.
Re: Forking not completing by merlyn (Sage) on Mar 15, 2001 at 03:52 UTC
Consider Mail::Bulkmail instead, although I always feel bad pointing out that module. It's ripe for spammers. -- Randal L. Schwartz, Perl hacker	[reply]
Re: Forking not completing by larryl (Monk) on Mar 15, 2001 at 06:33 UTC
You don't mention what `$mailprog` is, but assuming it's sendmail you might want to investigate the `-odq` option. If sendmail is running as a daemon on your system (look for something like `/usr/lib/sendmail -bd -q15m` in the process list), adding that option will not let the mail be delivered directly but instead queues it up for the next regularly scheduled time that sendmail will run the queue (e.g. `-q15m` means every 15 minutes). I'm not a `sendmail` guru, but it's my understanding that this will be more resource-friendly than starting up a synchronous sendmail process for each outgoing email.	[reply] [d/l] [select]
Re: Forking not completing by McD (Chaplain) on Mar 15, 2001 at 04:12 UTC
Interesting approach - for no good reason other than it's a fairly aggresive scheme, I'd shy away from that many forks - you'll feel the impact on that box of 400 processes sending mail. (So, too, will your mail daemon!) That said, I suspect you're hitting a specific limit of the OS and/or use that your CGI is running as. I suggest adding code to complain if the `fork` fails and see if that offers any clues. `if ($pid = fork) { # I'm the parent } else { if (! defined($pid) ) { die "Can't fork: $!"; } # Child here }` [download] Peace, -McD	[reply] [d/l] [select]
Re: Forking not completing by skazat (Chaplain) on Mar 15, 2001 at 04:27 UTC
This looks like code straight from Subscribe Me! Lite, I may be wrong though. The amount of processes isn't an issue, since its being forked once, and then mail is being sent with the foreach loop. I wonder if Sendmail is complaining that its getting way too many emails being sent at once and the whole thing is dying. Check the error logs. Sendmail isn't keen on sending mass emails, you may want to check out the Mail::Bulkmail package, or add some kind of batching function, where a small amount of email messages are sent in chunks. I feel stupid to suggest something I've made, but Mojo Mail has both the batching capability and the Mail::Bulkmail module already in there, so you can send mail using the sendmail program or a straight SMTP connection. Its also strickly for non-spammers, I use it to announce art projects of mine :) -justin simoni !skazat!	[reply]
Re: Forking not completing by arturo (Vicar) on Mar 15, 2001 at 03:54 UTC
I'm no expert on the use of `fork`, so I'm just going to suggest a different approach: rather than spawn off a further subprocess from the web server, why not have the CGI write to a file (say, the list of addresses and the message) and have a cron script (say, that runs every five minutes) check for the existence of that file and do the work then? Philosophy can be made out of anything. Or less -- Jerry A. Fodor	[reply] [d/l]
(tye)Re: Forking not completing by tye (Sage) on Mar 15, 2001 at 19:34 UTC
A guess: You don't "daemonize" the child process so it is still connected to the web server and so probably eventually gets killed when the web server decides that it is running too long. (This has to do with process groups and is, of course, web server dependent.) Update: Thanks to arturo for pointing out my usual spelling error. - tye (but my friends call me "Tye")	[reply]
Re: (tye)Re: Forking not completing by Anonymous Monk on Mar 16, 2001 at 01:19 UTC
how would you daemonize the process?	[reply]
(tye)Re2: Forking not completing by tye (Sage) on Mar 16, 2001 at 01:46 UTC
Use Super Search or search your *.pod documentation that came with Perl for the word "daemonize" for some details. - tye (but my friends call me "Tye")	[reply]