Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

You may find this comparison of MTAs interesting. Yes it is dated but Exim was slow as a dog back then. Here is another comparison which is a bit less dated and still shows Exim at the back of the field.

Now the hardware used was relatively old and slow but this is not a hardware bottleneck task. It is concurrency, network connectivity, and DNS lookups that are bottlenecks. To send an email your server has to look up the address in DNS (you want a big local cache), then make the connection to the remote server (may take seconds), then send HELO,MAIL,RCPT,DATA,EOM - all of which can be slowed down by the remote server. The entire transaction can easily exceed several seconds, thus you require multiple concurrent processes (50+) in your MTA to get any sort of decent throughput.

You will note that performace in terms of emails per second is a feeble 5-15 depending on MTA. At 20/second througput you are looking at 20 hours runtime for 1.5 million emails.

I suggest you look at a dedicated mail server that also runs its own DNS with a huge cache to do this task. Exim looks like one of the less efficient options judging from the performance benchmarks. If you dump 1.5 million emails onto your Exim queue and it is really only running a throughput of 5/sec you will effectively freeze outgoing email for 80 hours as any new message will go to the back of the queue. Even if it is doing 50/sec there will be nothing else going out for 8 hours - unless of course there is some way to flag it as low priority. Not only will you have an outgoing email problem you will also have an incoming email issue as Exim is also recieving incoming connections.

If you did want to split up the flatfile of email addresses you don't need a database. Just use split(1).


In reply to Re: Applying the brakes by tachyon-II
in thread Applying the brakes by Ryszard

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2024-04-19 09:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found