erniep has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,
I have a cgi program which takes a text file of 500 email address' and sends a single email message using the sendmail program.
This program is only used about once a week, but the ISP is complaining about the volume of mail.
My Question is : How could I send this email in such a way that there is no drain on the ISP's resources.
I am far from being an expert on using email but one resolution I have thought about is to send bundles of 20 or so emails every few minutes.
I'm sure some of my wise monk friends have had this problem and solved it in a more effective way.
Any help is always greatly appreciated.

Replies are listed 'Best First'.
Re: Bulk Email
by Juerd (Abbot) on Apr 18, 2004 at 12:38 UTC

    Have you seen Mail::Bulkmail? The module itself doesn't always work, but you can at least learn about bundling mail in envelopes :) The trick is that if you have 50 foo.com addresses and the email itself is identical, you don't need 50 connections.

    But if 500 messages is causing a problem, consider looking for another ISP or skip the ISP and send things directly over SMTP (install a local mail server and use it.)

    Juerd # { site => 'juerd.nl', plp_site => 'plp.juerd.nl', do_not_use => 'spamtrap' }

Re: Bulk Email
by skx (Parson) on Apr 18, 2004 at 12:12 UTC

    The obvious way to do this would be to sort your email addresses, and then mail them in small batches.

    I used to run a script which notified livejournal users of changes to their friends lists - this sent a mail a day if there was a change, and on average used to send 2000 mails.

    I set the script up so that rather than sending 2000 mails en masse, it would sort them and I had some cron jobs setup to process 'a-k', 'k-o', and 'o-z' individually.

    This way rather than one big burst you only had three smaller ones.

    If this isn't appropriate for you you could do something kludgy like "mail user; sleep(60); mail next user".

    Steve
    ---
    steve.org.uk
      While your sending volume is tiny, and while you're mailing an opt-in list, nonetheless it wouldn't hurt to order your emails in such a way so that they don't sorted mail by TLD... that is, don't mail all the @aol.com at once. Perhaps a random sort.

      Again, this matters little for 500 emails, but I wanted to raise the issue that 'not mailing too fast' has implications for both the sender side and the recipient side.

      (Of course, for really high-volume mailing, you establish relationships w/ AOL, MSN, etc and give them the heads up before bulk mailing into their system....)

Re: Bulk Email
by b10m (Vicar) on Apr 18, 2004 at 12:23 UTC

    Are you sending 500 emails, or 1 email with 500 BCC addresses (or 5 emails with 100 BCC addresses)? The latter approach will at least not open 500 sendmail processes trying to queue the emails on your ISP's server, and thus be a little nicer on your ISP (if I get it right ;-). Quite frankly, I don't think 500 emails is that much for an ISP -taken for granted that these are non-spam mails.

    By the way, have you looked into already existing mailing list options, such as majordomo (which is written in Perl :)?

    --
    b10m

    All code is usually tested, but rarely trusted.
      I think the OP is fudging numbers a bit, I do not know of one ISP that would baulk on 500 emails generated and queued at the same time. Make that 50,000 or >, or a large portion that reply to abuse out of 500 and you would get ISP issues. Either way it sounds more like spam and less like a mailing list to me...


      -Waswas
        Nah, activist friends of mine have had issues with their ISPs when they send to more than a hundred people at a time. Not 500 but 100.
        Once again I state this was a legitimate request from a community volunteer. You assume the worst in people and that's a shame.
        All programmers are not wicked people, some of us only want to help our fellow human beings.
        I recieve no compensation for my work with my community, I do it as a hobby after working 30 years in the computer business.
        Let's all try to give people the benefit of the doubt.
        From my favorite poem: "Exercise caution in your business affairs; for the world is full of trickery. But let this not blind you to what virtue there is; many persons strive for high ideals; and everywhere life is full of heroism.
Re: Bulk Email
by matija (Priest) on Apr 18, 2004 at 12:14 UTC
    That is certainly a good option - and talking things over with your ISP would give you the general idea about how many minutes there should be between the bundles.

    An alternative would be to either set up your own MTA (which would send mail directly to addressees) or modifying your script to use Net::SMTP to do the same thing. The later is not trivial at all (since you will need to implement MX lookups, retries and non-deliveries), so I would still suggest you go with the first option.

Re: Bulk Email
by dws (Chancellor) on Apr 18, 2004 at 18:47 UTC

    This program is only used about once a week, but the ISP is complaining about the volume of mail.

    I manage the update mailings for a conference, and have a similar challenge, though at a larger (5x) scale. Our monthly emails aren't time critical, so having the mailing spread out over 24 hours, sent in batches of 25 emails several times an hour, works just fine. Sorting the mailing list alphabetically introduced enough variation to ensure that a single batch couldn't contain more than 5 emails bound for any single domain. This reduces the chance that some heavy-handed email filter will cry "spam", forcing me to jump through time-consuming hoops with some ISP so that the folks on or opt-in mailing list will continue to get their updates.

Re: Bulk Email
by Happy-the-monk (Canon) on Apr 18, 2004 at 12:12 UTC

    How could I send this email in such a way that there is no drain on the ISP's resources. I am far from being an expert on using email [...]

    Bulk mail is something nobody likes to receive... (and depending on the circumstances and the area it comes from or goes to, it might even be illegal) so this is the one thing that makes the monks reluctant to answer.

    The other is: you offered a solution already, so what is the actual Perl question about it?

    Update:
    Yikes, I've drawn a flame. I am sorry!
    I neither mean to moralize nor question the good intentions of the OPoster. I state that the impression the OPosting left on me was a negative one. And while the OP may be no spammer, spammers read and learn the same way we do.

    It must be alright to ask about the perl question on Perl Monks, being eager to learn about it and answer it, if I'am able to. No harm was intended.

      Bulk mail is something nobody likes to receive... (and depending on the circumstances and the area it comes from or goes to, it might even be illegal) so this is the one thing that makes the monks reluctant to answer.

      What makes you say this? I take it you generalize all bulk e-mail and degrade it to "spam" (which a lot of people don't like to receive, correct). There are however mailing lists to which people subscribe. I don't think everybody on a mailing list dislikes receiving those e-mails. Please don't confuse bulk e-mail with UBE (Unsolicited Bulk E-Mail).

      --
      b10m

      All code is usually tested, but rarely trusted.

        It doesn't look like it's a mailing list the OP is asking for.
        Otherwise, you're correct, there must be people out there who like to receive mailing list email.

      Bulk mail is something nobody likes to receive...
      The other is: you offered a solution already, so what is the actual Perl question about it?

      The question was asked because I live in a community and volunteer my time to help with the community Internet site.
      It was asked in order to comply with the wishes of the ISP and asked for a resonable solution to a reasonable ISP concern.
      The email address' I refer to are from community residents that have asked to be updated on community affairs and/or emergencies.
      I thank the monks that have offer their advice as to how to solve the problem.

      I offered my potential solution and asked if there are alternatives, this seems to me to be reasonable .
      Update
      Mr. Happy--
      I thank you for your explanation. I agree that there are many many programmers out in cyberland whose motives are not positive. but perhaps we should think of the adage: "innocent until".
      Monks Rule...Thank you all once again