skazat has asked for the wisdom of the Perl Monks concerning the following question:

Look at all you wonderful people!

I want to play around with the sleep() function for a cgi script I release, but before I shoot myself in the foot, I have a few questions about it and posibly about how webservers may treat a sleeping process executed via a cgi script.

Is there any limit that can be put on how long a process can sleep before the webserver or perl itself goes "hey you know, this guy aint waking up any time soon, I'm gonna well, kill it before the whole system tweaks out"

I'd hate to have something seem to work, and then wig out halfway through. I don't know enough about configuring perl or Apache to know if something like this can be controlled. My guess is there is. If so, is there a default? Is there a safe way to check?

I'm using this for sending mailing list messages (no no no, I aint no spammer, this thing is used for real things.example -> http://quotes.prolix.nu/mailing_list.html) I'm just using sendmail or .qmail to send the mail messages but guess what? sendmail doesn't like when I , say, send 50,000 messages at once :) (I don't blame it)

So it seems that if I send mailings in batches, say, send 100 messages (sleep() an hour) send another 100 messages (sleep and hour) etc. we got a solution. Or possible even better send 1 message sleep 2 seconds send another message, sleep 2 seconds.

I'm looking into the Bulkmail module, but its possible that people who use this script won't have SMTP access, so I want them to be able to ue something else as a default.

Comments? Ideas?

-justin simoni
!skazat!

  • Comment on Getting Close and Cuddly with the sleep() function

Replies are listed 'Best First'.
Re: Getting Close and Cuddly with the sleep() function
by extremely (Priest) on Nov 06, 2000 at 03:29 UTC

    Fork off grandchildren to do the mailing, and return from the CGI immediately. Give em a URL that can check up on the progress of the children if you want to get fancy. A LOT of clients will time out at two minutes, a lot of IP stacks at 2-5 minutes, apache is pretty stubborn but it too will freak after a while.

    Making a CGI user wait for the status of a serious bulkmail is nuts. And worse, if they hit esc and reload to try and "get it to work" well then you had better be debouncing multiple submits or your mail users aren't going to be happy.

    extremely <== has done all this wrong before =)

    --
    $you = new YOU;
    honk() if $you->love(perl)

      >>And worse, if they hit esc and reload to try and "get it to work"
      >>well then you had better be debouncing multiple submits or
      >>your mail users aren't going to be happy.

      Each list made has a database with all sorts of good info in it. Its not hard to put a sending value at "1" and then only allow sending again when thats at 0. I have a pretty tech system going here, but to be honest, the worst part of it is the mailing part :)

      Although I believe nows the time to get that up to speed. -justin simoni
      !skazat!

Re: Getting Close and Cuddly with the sleep() function
by cianoz (Friar) on Nov 06, 2000 at 04:17 UTC
    as pointed out by extremely and roberto you should use fork() to spawn a new job and send your email.
    you should also close STDIN and redirect STDOUT and STDERR
    a little example:
    #!/usr/bin/perl -wT use strict; use CGI; if(fork()) { my $q = new CGI; print $q->header(); print $q->start_html(); ## print your html here print $q->end_html; } else { close STDIN; open(STDOUT, ">>/tmp/yourscript.log") || die "Can't redirect stdou +t"; open(STDERR, ">&STDOUT") || die "Can't dup stdout"; #you have to log warnings somewhere... select(STDERR); $| = 1; # so you can see what's appening with tail... select(STDOUT); $| = 1; #your email stuff here... }
    Update: Maybe you should also use a semaphore to prevent other istances of the script from running simultaneously...

      what does:

      if(fork())

      do? does this fork the process if it can?

      is fork not allowed in some systems? this script is most likely going to be running on *nix, I'm very unknowing about how to make any forked processes. There are no easy to understand tutorial on it :)

      would it be better to have the web-based program actually gather the information, then pipe this info to another script that only does the mailing? this seems to be a somewhat the same idea to fork() to a child process to do the dirty work. Then would that script not fall into the timeouts of the cgi program?

      >>you should also close STDIN
      probably a good idea!

      Seems simple to just mail whoever the owner of the list is every time a batch goes out, or when the mailing is done to keep tabs.

      I'm stil a bit shaky on how to do this, does anyone else have any good advice? -justin simoni
      !skazat!

        if(fork()) doesn't mean "if fork worked": :)))
        it means if fork return a true value (the pid of the child process) then i'm the parent and i have to print my html else i'm the children (fork return 0) and i have to send the email. see "perldoc -f fork" for more info (and some hints on how andle errors)
        fork is a unix system call (see fork(2)) but recent versions of perl try to implement it on windows too
        for the second question: if you pipe your info to another script you have to wait for the second script to finish (unless you fork before the pipe :-)
Re: Getting Close and Cuddly with the sleep() function
by roberto (Acolyte) on Nov 06, 2000 at 03:31 UTC
    This shouldn't be done from inside a CGI script, use a cron job or fork() a new child process. (it's not only a timeout issue: you will hang the client until the process ends or get killed..)
RE (tilly) 1: Getting Close and Cuddly with the sleep() function
by tilly (Archbishop) on Nov 06, 2000 at 05:12 UTC
    If you are on a Unix system, try man at. Just write out a file, fire off a long list of jobs to a queue, then return. Each job has a sleep built in. The last one clears the file.

    That will work well even if 10 people send off batches at once. It all gets queued up, then the queue empties one at a time.

Re: Getting Close and Cuddly with the sleep() function
by Malach (Scribe) on Nov 06, 2000 at 03:34 UTC

    Why, in particular is this a CGI script?

    The way I'd do it, if I had to script to send that many messages, and only had a web front end to do it from, would be to use the web system to generate the information, and write the information out to a temporary file... then have something else (perhaps in cron?) that checks for the existance of /path/to/that/file.info and starts sending the mail.

    Of course, if this is on a box-not-your-own, there are security issues with this... be aware.

    IANAPG, YMMV, etc

      >>Why, in particular is this a CGI script? Its a web-based Mailing List Manager :)

      ther main system is all web-based. I'm not geeky enough to make a terminal based, mailing list manager, nor are the people who like this script want one either,

      Theres no way i'm making people set a cron tab. I for one don't know enough about cron to do this, and some servers don't permit people to do this. cron is too geeky. A big green button that says "send" is easy to understand.

      -justin simoni
      !skazat!