You could have a look at Mail::Bulkmail which has some ideas on sending out bulk newsletters. It allows you to generate dynamic messages in bulk. I have used it in the past for sending out newsletters where the initial message is created using MIME::Lite, and personalized information is placed into the message by Mail::Bulkmail just before the message is sent out.

active_send_mail = 0
threshold = 20

for (each email in the list) {
  send_email(address, content)

  # make sure we don't fork too many processes
  active_send_mail = active_send_mail + 1
  if (active_send_mail >= threshold) {
    waitpid -1
    active_send_mail --
    report on status
  }
}

sub send_email
{
  fork
  if child {
    send_the_email using mail or Net::SMTP
    exit
  }
  # in parent
  return child pid
}
[download]

Actually, I really doubt this will be fast. In fact, it's likely to become a bottleneck itself. You're chewing up all the CPU's to try to load up the SMTP server with emails, leaving very little left for the SMTP server to use (if it's on the same machine), or leaving very little bandwidth for the SMTP server to send out the emails (if it's on a different machine).

Better bet is to use the envelope to specify all your recipients and send the mail out once. The SMTP server will only have to process a single message - which it will fork off and send on its own. For example, if 15 recipients are local, they can all be written through immediately without being choked by more incomings. And if another 5 are on another single host, the SMTP server can relay all of those in a single envelope, reducing both bandwidth and CPU time on both machines.

The only faster way than this is to send each mail to the proper machine directly - and, again, you need to send the mail to each host only once, using the envelope to address to multiple individuals at once on the same host to reduce the number of connections you need to make. But now you're writing your own SMTP server, which is unlikely to be a great use of your time.

Although you are correct that using the envelope to send the same message to multiple recipients on a single mail server is much more efficient, it does not solve the users problem, since it was specifically stated that all messages have dynamic content.

I would also agree that you should let the mail server do the hard work for you. If you want your program to run quickly, then tell your mailer to queue the message for later delivery. It will return immediately, and be ready for the next message. At some later point (seconds or minutes later), a queue runner will pickup the waiting messages and send them out. This means your program can run quickly, even though the messages will be limited in the speed that your mail server can send them out.

If you are using qmail, using qmail-inject will do this automatically. If using sendmail, I believe sendmail -ob (or maybe -oq) will do it. If you are sending directly through SMTP, then of course this technique doesn't apply.

Actually, I really doubt this will be fast. In fact, it's likely to become a bottleneck itself. You're chewing up all the CPU's to try to load up the SMTP server with emails, leaving very little left for the SMTP server to use (if it's on the same machine), or leaving very little bandwidth for the SMTP server to send out the emails (if it's on a different machine).

Presumably you would set threshold to a suitable value to keep the SMTP server going as efficiently as possible. After initially loading up the SMTP server with work to do, the script should spend most of its time waiting.

This looks ok for me as long as you don't use SMTP to send the messages.
If you do use SMTP, forking and using multiple connection will be slower than use just one connection and eject all the email in the main process.

I think.
TheMage
http://themage.bliker.com

I don't really know how slow/fast it is - as I still don't have large list (just few emails, mostly mine and few friends) that I use for testing.

Anyway, in the control panel of application, when you create an email and place it in outbox, you can select the date & time when to send that particular email. I did it that way, so someone could prepare say several issues of newsletter, before they are to be sent. Obviously cron is set to run every hour or so, a script that checks if there are some emails in outbox that need to be sent. And there is no direct action performed by user ...

So to answer your question - no my goal is not to notify when all emails are delivered (it's an option - it sends email when it finishes if you want it of course).

And by my logic - if my application forwarded it to the SMTP server then my part of job is done - so it can send the notification.

If this speed-up can only be done by using qmail/sendmail type applications and not SMTP server directly - then it's fine with me. In my features I will write the opposite. Kind of : "If you use Open Source OS you get a special bonus - faster sending of emails." :)