frank1 has asked for the wisdom of the Perl Monks concerning the following question:

1). I need help with sending emails with their names in email body

2). Can i just loop to make sure each recipient gets email or Sub is enough. to send to over 100k emails

DB ID name email 1 jane test@test.com 2 tom tom@tom.com
my $msg = "Hello 'name' welcome to earth"; my $query = $dbh->prepare("SELECT DISTINCT email FROM users"); $query->execute(); my $records = $query->fetchall_arrayref; my @to; for my $em ( @$records ) { push @to, @$em; } for my $recipient(@to) { mail($recipient); } sub mail { my $recipient = shift; my $smtpserver = 'smtp.dddd.com'; my $smtpport = 587; my $smtpuser = ''; my $smtppassword = ''; my $transport = Email::Sender::Transport::SMTP->new({ host => $smtpserver, ssl => 'starttls', port => $smtpport, sasl_username => $smtpuser, sasl_password => $smtppassword, }); my $email = Email::Simple->create( header => [ To => $recipient, From => 'dddd@ddd.com', Subject => $sub, 'Content-Type' => 'text/html', ], body => $msg, ); sendmail($email, { transport => $transport }); my $Response = "Success!"; }

Replies are listed 'Best First'.
Re: Send email with recipient name
by Corion (Patriarch) on Nov 21, 2023 at 15:47 UTC

    Sending 100k+ emails will most likely get smtp.ddd.com on a lot of blacklists.

    At the minimum you need to throttle sending mails, and most likely you will also need to handle rejections properly.

    You should also make sure that your SMTP server at smtp.dddd.com can handle 100k+ queued mails.

    Personally, I would add provisions to restart the script from a given mail address. This involves sorting the list of mail addresses and then selecting all mails after the last (successful) mail.

    Note also that you are (re)creating your $transport for every mail you try to send. You might want to do that outside of your mail subroutine instead.

    Again, sending 100k+ mails sounds like a really bad idea and you should think about using one of the mail sender services like Amazon SQS or Mailchimp or whatever.

      That's all good advice, but it was already given in one of the previous threads. Perhaps repetition will hammer the points home, though.


      🦛

        I hear things won't get exciting until the 6th thread, wake me up when that happens.

      Sending 100k+ emails will most likely get smtp.ddd.com on a lot of blacklists.

      Am using email smtp provider, like Amazon SES, Sendgride & others.

      So my main point is to develop my script to do what i want. because am using my smtp server providers. to monitor deliveries and everything.

      So i dont have to worry about smtp server issues, i just want to develop my script to deliver all emails with their names in email body

        First, most of the email providers give you a way to form-fill an email template. So, you can just log into your admin portal, type out the form mail, then look at their API to find out how to deliver the list of addresses and names. That's the easiest way.

        If you really want to use SMTP, maybe for reasons you haven't told us, then like corion mentioned, you first should think of how to resume the mailing after it gets interrupted, because it would be bad (and expensive) to have to re-start from the beginning of the list and send some of them twice. You can either sort the emails and log as you send them so that you know where to pick up from, or you could write your progress to a database table in a way that you can later select only the email addresses that haven't been sent the email yet. Another method is to make a "spool" table where you queue one record for each email that you want to send, then delete one row from that table after each email is sent. This lets you separate the seteps of preparing the data to email, and delivering the emails, so you can inspect the results of the first phase before you have sent anything, and if something is wrong you just delete all the records and try again. When the queue looks accurate, then you begin sending, and deleting the record or updating it with the results of the send attempt.

        In your code, this looks suspicious:

        my @to; for my $em ( @$records ) { push @to, @$em; }
        It looks like each element of $records is an array of *fields*, right? with one field being the email and one field being the name? I would think you would just do
        # Slice makes it return an arrayref of hashrefs my $records = $query->fetchall_arrayref({}); my $transport= ... for my $record (@$records) { mail($from, $record->{to}, $subject, $record->{name}); } sub mail { my ($from, $to, $subject, $name)= @_; my $html= .... # form-fill the name into an html template my $email= ... sendmail($email, { transport => $transport }); }

        You should definitely use a persistent mail transport. Here's a snippet from one of my systems:

        $transport= Email::Sender::Transport::SMTP::Persistent->new( host => $host, ssl => ($port == 465)? 1 : ($port == 587)? 'starttls' : 0, port => $port, $user? (sasl_username => $user) : (), $pass? (sasl_password => $pass) : (), debug => $log->is_trace, );

        That makes one SSL connection to the server and then sends all the mail through it without needing to re-initialize the SSL each time. It's massively faster than re-connecting for each email.

        If you're looking for a good way to assemble a quality email message, I recommend Email::MIME::CreateHTML. Here are some other snippets you might find helpful:

        # Assuming you start with an HTML document for your email: $text= HTML::FormatText::WithLinks->new(unique_links => 1) ->parse($html);
        (it's good practice to include the text-only version of your html email, and probably helps not getting flagged as spam)
        # Because most mail clients don't properly process <style> blocks, # we have to embed the style onto each individual element. # This class does the job for us. my $inliner= CSS::Inliner->new(); $inliner->read({ html => $html, charset => 'utf8' }); $html= $inliner->inlinify;

        I forget why my code is calling CSS::Inliner when it also sets inline_css => 1 in the options to create_html below. I'll let you discover if that's necessary :-)

        # Build the email. The resolver will find all external images # and stylesheets and suck them in as multiple parts. my $email= Email::MIME->create_html( header => [ ], # we use the unicode-enabled setters, instead text_body_attributes => { charset => 'UTF-8', encoding => 'quoted-printable' }, text_body => encode('UTF-8', $text), # text gets assigned as-is body_attributes => { charset => 'UTF-8', encoding => 'quoted-printable' }, body => $html, # body gets assigned as body_str embed => 1, base => "https://YOUR_WEBSITE", # Prevent excessive lookups of resources object_cache => $self->resource_cache, inline_css => 1, %$args, ); # These methods properly handle unicode, which is why we use them # instead of passing these to the constructor $email->header_str_set(To => (ref $to eq 'ARRAY'? join(', ',@$to) : ''.$to)); $email->header_str_set(From => (ref $from eq 'ARRAY'? join(', ',@$from) : ''.$from)); $email->header_str_set(Subject => ''.$subject);

        Note all the details about utf-8. Sadly Email::MIME::CreateHTML doesn't have nice support for that yet. Email::MIME doesn't really either - I mean the options and methods exist, they just aren't the defaults.

Re: Send email with recipient name
by haj (Vicar) on Nov 21, 2023 at 16:38 UTC

    100k emails... that's a lot.

    While this is technically possible, I can hardly imagine a legitimate use case to collect that many addresses in a database - and not using a CRM system which deals with mail distribution. Are you really sure you need to roll out your own?

    Also, if the name is the only part of the text which is different, then I must ask: Are you sure that the recipients welcome that message you're sending?

      As a developer/sysadmin of a system that sends 80K/day, 100K isn't that much. I mean nobody wants to help a spammer, but if you consider that there are 300M people in the USA and probably 2/3 of them have an email, and half of that probably have multiple email addresses, and then extrapolate to the English-speaking world, a typical spammer is probably targeting billions, not a measly 100K. And who knows, frank1 could actually be operating in India or China or something.

      Honestly, 100K email/day is probably the point where you want to roll your own. My client switched from their own to SendGrid back when they were doing maybe 40K/day and their SendGrid bill is fairly expensive and has doubled since then. SendGrid does some work for you to keep your system off blacklists, but at the same time at this volume you need a dedicated IP address for reputation control, and that IP address could just as easily be a permanent IP at a data center for a lot less $/mo, and the monthly cost could easily pay for the time for a developer to set up all the SPF and DKIM and "warm up" the IP address and inactivate the accounts of the bounces, and occasionally respond to spam requests. So in retrospect, I think it was probably a mistake to move to SendGrid for such high volumes. SendGrid and MailChimp are really more for the smaller apps that send a few hundred a day, where the effort of maintaining your own mail server would be much more expensive than the few cents you pay per email.

      Are you really sure you need to roll out your own?

      Yes

      Are you sure that the recipients welcome that message you're sending?

      Yes, because these are customers who registered and agreed to terms and conditions of receiving daily notices from system