insite2000 has asked for the wisdom of the Perl Monks concerning the following question:

On our shared web hosting account, we are limited to 600 outbound e-mails per hour (subscribers only, we're a spam free zone). We have several thousand people in our database who need to receive our publication every day. Since 500 e-mails go out in under 10 minutes, Exim gets crabby and bounces back a bunch of "unroutable mail domain" messages and the they don't get through to subscribers. One possible solution is to control the number of e-mails per hour that we are sending, and I'm wondering if sleep is the correct approach. Here's the code block where I'm thinking of placing it. Is this the right thing to do? 
foreach (@{$userinfo}) {
  $sendmail_info->{EmailAddress} = $_->{EmailAddress};
  SendIt($sendmail_info);
  # Here is where I think we should say
      sleep 15; 
  # (or other interval, 4 emails per 
  # minute equals 240 e-mails per hour)
}

sub SendIt {
  my $info  = shift @_;
  my $msg = MIME::Lite->new(
     'From'    =>'foo@foobar.com'
     'To'      => $info->{EmailAddress},        
     'Subject' => $info->{EmailSubject},    
     'Type'    =>'multipart/mixed'
  );
  
   $msg->attach(Type =>'TEXT',
        Data => $info->{EmailMessage}
        );
   $msg->attach(Type        =>$info->{MimeType},
        Path        => $info->{Attachment},
        Filename    => $info->{AttachmentName},
        Disposition => 'attachment'
        );
   
   $msg->send_by_sendmail('SetSender'=>1,'FromSender'=>'foo@foobar.com
+');
}

[download]
I bet there's a Monk out there who recognizes this. :-)

Replies are listed 'Best First'.
Re: Controlling e-mails sent per hour
by rob_au (Abbot) on Feb 06, 2004 at 05:26 UTC
    One change which I would make would be the following which implements a variable sleep such that the sending of mail is more evenly timed to match 15 second intervals - I have used something similar for a dispatch of mail to our (many thousand) local users in order to stagger the load on the mail server. 

    eval "use Time::HiRes qw( time sleep )";

foreach (@{ $userinfo }) {

    my $time = time();

    ...

    sleep 15 - ((time() - $time) % 15);
}

    [download]

    However, if you are employing sendmail as a local MTA, you may be interested in taking a look at vthrottle which is a rate throttling mechanism for Sendmail implemented via the milter interface (which can also be interfaced in Perl by way of Sendmail::Milter or PMilter). This may constitute a longer term solution for your mail throttling.

     

    perl -le "print unpack'N', pack'B32', '00000000000000000000001011000000'"

[reply]
[d/l]
[select]
•Re: Controlling e-mails sent per hour
by merlyn (Sage) on Feb 06, 2004 at 13:09 UTC
    Having read the rest of the answers so far, let me suggest that a fork every six seconds (600 per hour) is not very expensive, so just fork to send each message. Then if one gets blocked, at least you're trying to send the next one six seconds later anyway.

    Or, set up a POE queue, and send the mail via a timer loop.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

[reply]
Re: Controlling e-mails sent per hour
by Ryszard (Priest) on Feb 06, 2004 at 10:14 UTC
    I have a similar situation whereby i need to send only two hundred emails, pause, another two hundred etc etc.

    I have a daemon that sleeps every 5 mins then kicks in to see if there are any "Bulk Mails" to send.

    Sleep works well for me in the daemon part where it scans for new mails every 300 seconds, and in the other part where it pauses after a run of 200 mails.

    No, no spam, we've a customer base of around 300k that needs explicit communication sometimes.

[reply]
Re: Controlling e-mails sent per hour
by Abigail-II (Bishop) on Feb 06, 2004 at 13:23 UTC
    My first step would be to contact the ISP and see whether they are willing (perhaps for a fee), to solve this 'problem' at the MTA level, by using a throttle. That's a far more flexible solution, that avoids clutter in your programs as well. It also automatically solve the problem of having two programs simultaniously trying to set a large batch mails. You also don't have to reconfigure your program(s) if you and your ISP agree on a different limit of mails/hour.

    It's an MTA level problem, and that's where it needs to be solved.

    Abigail

[reply]
      I think the OP said it is a "Shared Web Hosting Account". For a service like this, you are in control of the throttle at that point. They just charge you for the additional bandwidth. They also setup monitoring tools to make sure that you are not a SPAMMER (which at 600 per hour would raise a red flag). I think the OP would have to solve this problem.
[reply]
Re: Controlling e-mails sent per hour
by z3d (Scribe) on Feb 06, 2004 at 10:04 UTC
    I'm not a 100% sure that sleep is your best bet in the end though, at least not as is. If for whatever reason there's a slight delay in being able to get those emails out, you could be losing maximum usage of your outbound mail. For instance, if it takes a few seconds to send a batch, sure, that's fine. But what if the mailserver is in a crabby mood that day and it takes a couple extra minutes to send? One off the cuff thought on this is that you could still use sleep, but maybe use a timer function (for some reason using benchmark comes to mind, but I'm sure that's not the most elegant solution) so that you can set your sleep to be a variable interval - maximum per time block minus how long it took to send the last batch, rounded up to an integer. Just a thought - of course, I live in a weird world where things never go as smoothly as I had hoped :)


    "I have never written bad code. There are merely unanticipated features."
[reply]
      somehow I missed rob_au's post before mine (time - yeah, need more coffee or something). sorry about the double post
[reply]
Re: Controlling e-mails sent per hour
by graff (Chancellor) on Feb 06, 2004 at 03:35 UTC
    I'm not the monk who recognizes your code, but I think you're basically right about sleep being the thing to use, and you have it in as good a place as any.
[reply]
      Thanks graff
[reply]
Re: Controlling e-mails sent per hour
by punchcard_don (Beadle) on Feb 06, 2004 at 13:06 UTC
    Opportunity to add a question to a question:

    What about cron? In the original post's specific situation, could you store the index of the last user to be emailed somewhere, then pick up from where you left off in the database every xx seconds with cron? Or will cron go down to that small a time unit?

[reply]
      Cron can only trigger once per minute.
[reply]

Back to Seekers of Perl Wisdom