JaredHess has asked for the wisdom of the Perl Monks concerning the following question:

Howdy Again Perl Monks!

I'm tasked with sending out emails to all of the subscribers to our company's email newsletter. I'm using the MIME::Lite module to allow HTML tags and a PDF attachment. I have created a sample script that seems to work fine overall, but I have two questions.

1) What is the most efficient way to send out a group email like this? We have about 500 subscribers currently. Right now I'm grabbing all the email addresses from our database putting them into an array and then I'm going through each item in the array and sending the email message out to each one.

sub sendemail { $j = 0; foreach $i(@names){ ## Grab user info from the table $name = $names[$j]; # Get their name $email = $emails[$j]; # Get their email $subkey = $subkeys[$j]; #Get their subscriber key $remotehost = $ENV{'HTTP_HOST'}; $remotehost = "http://" . $remotehost . "/"; $locunsubscribe = 'perl/nl_unsubscribe.pl' . '?' . $subkey; $locunsubscribe = $remotehost . $locunsubscribe; mailit(); $j++; } } sub mailit{ print "<p>...Attempting to send message to user:"; print "<H4>Record for $user</H4>"; print "<UL>"; print "<LI>User: $user"; print "<LI>Name: $name"; print "<LI>Email: $email"; print "</UL>"; print "<HR>"; my $wai = 'Wilcox Associates Inc.<admin@wilcoxassoc.com>'; my $when = localtime(); my $locwai = '../Contactinfo/feedback.htm'; my $mail_host = 'mail.wilcoxassoc.com'; # Replace w/ browse button on form to get these. my $fullpath = 'd:\temp\test.pdf'; my $filename = 'test.pdf'; ### Create the multipart container $msg = MIME::Lite->new ( From => $wai, To => $email, Subject => $subject, Type =>'multipart/mixed' ) or die "Error creating multipart container: $!\n"; ### Add the text message part $msg->attach ( Type => 'TEXT/HTML', Data => "<p>Dear $name,</p><p>$body</p><p></p><p></p>To unsubscribe +from the WAI Newsletter, follow this link: $locunsubscribe<hr>THIS IS AN AUTOMATED MESSAGE! PLEASE DON'T REPLY TO + THIS EMAIL." ) or die "Error adding the text message part: $!\n"; ### Add the specified PDF file $msg->attach ( Type => 'application/pdf', Path => $fullpath, Filename => $filename, Disposition => 'attachment' ) or die "Error adding $fullpath: $!\n"; ### Send the Message MIME::Lite->send('smtp', $mail_host, Timeout=>60); $msg->send or push @notsent,$email; }
I just don't know if this is the most efficient way. Is there a way to group all the emails together into one single message? For example, in Outlook you can create a group of email addresses and it sends out a single email to each address. Or does my route essentially do the same thing?

Also, when I run my script on my local server, I get an message from my Perl editor (OptiPerl) that shows an email and has this message:

SMTP RCPT command failed: <email@here.com>: Relay access denied

and it points to this code: $msg->send or push @notsent,$email;

But my script never finishes. It apparently dies after sending out all the emails but before it goes into my results subroutine here:

# Show the results of the send. This sub checks to # see if the array contains any addresses that it coulnd't send to and + displays them: sub showresults{ print "<h3>Showing Results</h3>"; if (@notsent) { print "<font color = 'red'><p>Out of the following emails:"; print "<br>@to"; print "<p>The following were not sent:"; print "<br>@notsent</font>"; } else { print "<font color = 'blue'><p>Emails successfully sent to the +se addresses:"; print "<br>@to</font>"; } print "<p><a href=$accountmanagement>Return to the Account Management +Area</a></p>"; print "<p><a href=$locscriptdownload>Visit the Script Repository</a></ +p>"; }


In this situation, I need it to be able to simply collect the email address that it couldn't send to the subscriber and continue with the script.

2) What code should I use to capture an email address that it couldn't send to and force it to continue the script? I'm using this line but I don't think that does what I'm hoping it would:
$msg->send or push @notsent,$email;


Below contains my entire script:
#!/usr/local/bin/perl # This script sends out an email to a specified group of individuals f +rom # the database use CGI qw(:all); use DBI; use Net::SMTP; use MIME::Lite; print header; #################### # PROGRAM OVERVIEW # #################### # Is user admin? If not kill process. isadmin(); # Display an email form. # - Subject # - Content # - Send # Clicking the send button will # check if all email fields are filled out. # And if they are, will process the data, and send the # email to the selected users. # If not, it will display the form again. if (param("subject") and param("body")) { # Get the array / hash of NL subscribers getsubscribers(); sendemail(); # Show the results of the routine. showresults (); } else { displayform(); } ######################## # SUBROUTINES ########## ######################## sub isadmin { $servername = $ENV{'SERVER_NAME'}; ## Sends Environement Variables if ($servername eq "localhost" or $servername eq "JaredWork"){ $remoteuser = 'admin'; } elsif ($servername eq "www.wilcoxassoc.com"){ $remoteuser = $ENV{'REMOTE_USER'}; } else{ foreach $key (sort(keys %ENV)){ print ("<p>$key = $ENV{$key}"); } &dienice("Couldn't connect to unknown servername of: $serv +ername"); } $remoteuser = lc($remoteuser); $datasource = 'dbi:mysql:userinfo'; $dbusername = 'perlscript'; $dbpassword = 'scriptpass'; ## Connect to database $dbh = DBI->connect($datasource,$dbusername,$dbpassword) or dienic +e ("Can't connect: $! ." . $DBI::errstr ); ## Grab user info from the table $sth = $dbh->prepare("SELECT * FROM registered WHERE username = ?") or dienice ("Couldn't prepare select state +ment: $!" . $dbh->errstr); $sth->execute ($remoteuser) or dienice ("Couldn't execute prepared + statement $!" . $dbh->errstr); @row = $sth->fetchrow_array(); $user = $row[0]; # Get their username $name = $row[1]; # Get their name $email = $row[2]; # Get their email $isadmin = $row[4]; # Get the admin column if (length($name)<=0) { dienice("There is no username that matches $remoteuser in the database. Use a different username."); } if ($isadmin ne "Y" and $isadmin ne "S") { dienice("$name, You do not have priveleges to send out a mailing to the users."); } } sub getsubscribers { $sth = $dbh->prepare("SELECT * FROM subscriber") or dienice ("C +ouldn't prepare select statement to get list of NL subscribers: $!" . + $dbh->errstr); $sth->execute() or dienice ("Couldn't execute prepared statement $ +!" . $dbh->errstr); while (@row = $sth->fetchrow_array) { $name = $row[2]; # Get their names $email = $row[1]; # Get their emails $subkey = $row[7]; # Get their subkeys # print "<p>Testing... $name, $email, $subkey"; push @names,$name; push @emails,$email; push @subkeys,$subkey; } } sub displayform { print "<h2>PC-DMIS Newsletter Subscribers - Email Form</h2>"; print "<p>Use this form to send out emails to newsletter subscribers. Simply type the subject of the email and the body into the form below. You don't need to include a 'Dear (name)' line. This is handled automatically:</p>"; # print "<p>Testing...list of emails to send to: @emails</p>"; $locscriptdownload='../scriptdownload.pl'; $accountmanagement='accountmanagement.pl'; print <<FORM; <form method="post" action=""> <p><b>Subject: </b> <input type="text" name="subject" size="50"> </p> <table width="100" border="1" height="403"> <tr> <td width="66%" bgcolor="#CCCCFF" height="357"> <p><b>Body:</b></p> <p align="center"> <textarea name="body" cols="50" rows="20"></textarea> </p> </td> </tr> </table> <p> <input type="submit" name="Submit" value="Send Now"> <input type="reset" name="Reset" value="Reset"> </p> </form> <p><a href="$accountmanagement">Return to the Account Management Are +a</a></p> FORM } sub sendemail { print "<h3>Entering Send Mail Routine</h3>"; print "<p>This email message will be sent:"; $body = param("body"); $subject = param("subject"); print "<P><b>Subject:</b> $subject"; print "<P><b>Body:</b> $body<hr>"; $j = 0; foreach $i(@names){ ## Grab user info from the table $name = $names[$j]; # Get their name $email = $emails[$j]; # Get their email $subkey = $subkeys[$j]; #Get their subscriber key $remotehost = $ENV{'HTTP_HOST'}; $remotehost = "http://" . $remotehost . "/"; $locunsubscribe = 'perl/nl_unsubscribe.pl' . '?' . $subkey; $locunsubscribe = $remotehost . $locunsubscribe; mailit(); $j++; } } sub mailit{ print "<p>...Attempting to send message to user:"; print "<H4>Record for $user</H4>"; print "<UL>"; print "<LI>User: $user"; print "<LI>Name: $name"; print "<LI>Email: $email"; print "</UL>"; print "<HR>"; my $wai = 'Wilcox Associates Inc.<admin@wilcoxassoc.com>'; my $when = localtime(); my $locwai = '../Contactinfo/feedback.htm'; my $mail_host = 'mail.wilcoxassoc.com'; # Replace w/ browse button on form to get these. my $fullpath = 'd:\temp\test.pdf'; my $filename = 'test.pdf'; ### Create the multipart container $msg = MIME::Lite->new ( From => $wai, To => $email, Subject => $subject, Type =>'multipart/mixed' ) or die "Error creating multipart container: $!\n"; ### Add the text message part $msg->attach ( Type => 'TEXT/HTML', Data => "<p>Dear $name,</p><p>$body</p><p></p><p></p>To unsubscribe +from the WAI Newsletter, follow this link: $locunsubscribe<hr>THIS IS AN AUTOMATED MESSAGE! PLEASE DON'T REPLY TO + THIS EMAIL." ) or die "Error adding the text message part: $!\n"; ### Add the specified PDF file $msg->attach ( Type => 'application/pdf', Path => $fullpath, Filename => $filename, Disposition => 'attachment' ) or die "Error adding $fullpath: $!\n"; ### Send the Message MIME::Lite->send('smtp', $mail_host, Timeout=>60); $msg->send or push @notsent,$email; } # Show the results of the send. This sub checks to # see if the array contains any addresses that it coulnd't send to and + displays them: sub showresults{ print "<h3>Showing Results</h3>"; if (@notsent) { print "<font color = 'red'><p>COULD NOT SEND TO the following +emails:"; print "<br>@notsent</font>"; } else { print "<font color = 'blue'><p>All emails were sent."; } print "<p><a href=$accountmanagement>Return to the Account Management +Area</a></p>"; print "<p><a href=$locscriptdownload>Visit the Script Repository</a></ +p>"; } #################### ### DYING NICELY ### sub dienice { my($errmsg) = @_; print <<DIENICE; <HTML><HEAD><TITLE>ERROR!</TITLE></HEAD> <BODY bgcolor="#CDDAE0" text="red" link="#000066" vlink="#6666 +66" alink="#CCCCCC"> <H1>ERROR...</H1> <P>$errmsg</P> </BODY> </HTML> DIENICE exit; }

Replies are listed 'Best First'.
Re: Most efficient way to send mass email?
by davido (Cardinal) on Jan 05, 2005 at 04:15 UTC

    You really need the right tool for the job. With respect to mass mailings (hopefully we're talking about opt-in mailings), MIME::Lite is a fish out of water, and Mail::Bulkmail is a great white shark. ;)

    The POD for Mail::Bulkmail has this to say:

    Mail::Bulkmail gives a fairly complete set of tools for managing mass-mailing lists. I initially wrote it because the tools I was using at the time were just too damn slow for mailing out to thousands of recipients. I keep working on it because it's reasonably popular and I enjoy it.

    In a nutshell, it allows you to rapidly transmit a message to a mailing list by zipping out the information to them via an SMTP relay (your own, of course). Subclasses provide the ability to use mail merges, dynamic messages, and anything else you can think of.

    And the Synopsis shows the following simple code example:

    use Mail::Bulkmail /path/to/conf.file my $bulk = Mail::Bulkmail->new( "LIST" => "~/my.list.txt", "From" => '"Jim Thomason"<jim@jimandkoka.com>', "Subject" => "This is a test message", "Message" => "Here is my test message" ) || die Mail::Bulkmail->error(); $bulk->bulkmail() || die $bulk->error;

    Hope this helps! Of course I only answered this question because I believe that your 500 recipients all want you sending them messages. If you're spamming people, let the shame of 27000 perlmonks rest on your shoulders. ;)


    Dave

Re: Most efficient way to send mass email?
by legato (Monk) on Jan 05, 2005 at 00:32 UTC

    Well, firstly, you haven't made a very good case for not using something free and extant like Mailman, which is designed explicitly to handle mailing lists, and which will likely scale far better than something hand-rolled.

    Secondly, you can group your addresses a bit, sending a message to 20 recipients at a time, and letting the e-mail server sort it out.

    $cnt = 0; @addr_slice = @addresses[$cnt..$cnt+20]; #Now, when you create your message: $msg = MIME::Lite->new ( From => $wai, To => join(', ',@addr_slice), Subject => $subject, Type =>'multipart/mixed' ) or die "Error creating multipart container: $!\n";

    But, I would work through a massive refactor of your code. It isn't fault-tolerant, there are a number of issues that would be caught by -w and use strict, and it demonstrates a lack of understanding of how mail systems and distribution lists work. I'm not being insulting, BTW: just suggesting that understanding more about how mail works would help improve your code.

    Better than a refactor -- use a tool that's already designed for this task, like Mailman.

    Anima Legato
    .oO all things connect through the motion of the mind

      @addr_slice = @addresses[$cnt..$cnt+20]; . . . . To => join(', ',@addr_slice),
      I assume that one of the things that the package you are advocating makes real easy is *not* exposing one person's email address to another person. Do *not* use anything like the above code and expect to be thanked for it. One of the good side-effects of the original code was that no recipient would discover anyone elses email address.

      While it does make a lot of sense to try to send to more than one recipient at a time, at least _try_ to use the Bcc: field to do so, not the To: or Cc: fields.

      BTW: the efficiency best comes into play if you can sort your addresses by destination host name (the "right-hand side" of the email address). This way it is possible to send one copy from your side to the remote SMTP server, where it is then the remote side that make and distributes copies to the several local recipients.

        Yet another excellent point. I guess the point of my code example was to show how quickly approaching this type task can get very messy, and to strongly suggest using something that has already been written and tested ad nauseum instead of reinventing a wheel.

        Your privacy point just reinforces my general point; thank you.

        Anima Legato
        .oO all things connect through the motion of the mind

Re: Most efficient way to send mass email?
by Anonymous Monk on Jan 05, 2005 at 10:36 UTC

    What is the most efficient way to send out a group email like this?

    By letting your MTA do the hard work. I'd do one of the following, in order of preference:

    1. Use dedicated mass-mailing software. (Mailing list software, not spam).
    2. Cheat (qmail solution): write the addresses to send to /var/qmail/aliases/.qmail-maillinglist (one address per line), then send an email to maillinglist@mydomain.
    3. Fork, opening a pipe between the two processes (using open "|-" or open "-|". Process that reads from the pipe does an exec "sendmail", @addresses (assuming the addresses to send to are in @addresses). Process that writes to the pipe composes the email, and prints it to the pipe handle. If you have more addresses than you can pass into exec, just splice the array in smaller parts and open various loops.

    What code should I use to capture an email address that it couldn't send to and force it to continue the script?

    No code. Undeliverable addresses is something for the MTA to worry about (retry, bounce, etc).

Re: Most efficient way to send mass email?
by fraktalisman (Hermit) on Jan 05, 2005 at 09:28 UTC
    For my company I wrote a script using MIME::Lite or, for simple mails, just sendmail. Maybe not the most efficient solution, but it works.
    The script is supposed to run on virtually any server with perl and SQL, so it does get used on webhosting servers that we didn't configure ourselves. Some of those don't even let us see the error logs.
    Some servers have introduced certain restrictions to prevent sp4mming and d.o.s. attacks:
    One server has a limit on how many mails can be sent by scripts per day. Another one does not allow more than a certain amount of mails per time - the helpdesk man told me I should slow down the script so it will not send more than one mail per second. Well, I did slow it down, but not that much. I also had to introduce a not very elegant way of checking sendmail's error output into another logfile. Maybe things the Bulkmail module would have done for me.
    But if you don't control the server, you have to take a second and third look at things and you can't take anything for granted. That's not nice, but it was also something to learn for future projects.
Re: Most efficient way to send mass email?
by JaredHess (Acolyte) on Jan 05, 2005 at 17:36 UTC
    Thanks for all the replies! I readily admit that I am not familiar with email systems or distribution lists--nor even Perl...

    Truth be told, I am not even a real programmer. In my job I mainly do technical writing (English graduate), but we're a small company so I also maintain our company's website, and from time-to-time this requires some CGI programming etc. All my Perl I picked up on my own and I know it's not elegent and there's probably a million better ways to do it. Most of the time I don't know what I'm doing, but I look at examples and eventually come up with a solution if I play around with it long enough. This was why I posted my questions, to become more familiar with it and see what others are doing.

    Why didn't I just go with existing mailing services? Before I even thought of using them I had already spent a lot of time collecting customer data via a form, building and maintaining a subscriber list complete with unique subscriber codes, unsubscribe scripts, etc. All customer data is stored in our own MYSQL database, and I don't know of a way to expose our collected data a service such as Mailman.

    And no, I'm not a spammer. . . (*insulted grunt*). If I was a spammer, I'm sure my script would already be much more efficient and cunning and I wouldn't be posting here. :-) This is simply an opt-in Newsletter.

    Anyway, thanks everyone! You've given me some direction. I will ponder upon these mysteries and attempt to come up with a course of action.

    Jared