Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re^3: Simple (but robust) email server (receiver)

by Tanktalus (Canon)
on Dec 16, 2008 at 15:46 UTC ( [id://730673]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Simple (but robust) email server (receiver)
in thread Simple (but robust) email server (receiver)

Creating a new perl process for every message is more resource-intensive in some ways, not in others.

First off, all the SMTP-handling code is in C. That's got to be less resource-intensive than perl. Second, it's already written, most bugs worked out, and someone else will fix any new bugs found. That's less resource-intensive than writing it yourself (you are a resource, too). Third, spawning off a new perl process, compiling and then running a bunch of perl code (both a .pl and a bunch of modules) is really not that resource-intensive. The only really expensive part here is the DB connect, which you can mitigate with FastCGI or DBD::Proxy if it really becomes a problem (premature optimisation--). Oh, and by using qmail, you get the preforking done for you. No, your code isn't forked - but if multiple emails come in at the same time, qmail will kick you off multiple times, providing with better scaling without any thought or design on your part. That, too, makes it less resource-intensive (again, you're a resource). And spewing the message over a pipe (purely in RAM, remember) is trivial - we're talking about copying a couple of KB around (usually). That's not resource-intensive at all, especially considering perl already does that type of work when you pass around scalars instead of references to scalars.

Personally, I like the "use qmail or postfix" solution as a starting point, as it allows you to focus on the real work you're trying to accomplish (stuffing a database) without worrying about stuff you don't really care about (SMTP, preforking, dropping privileges, etc.). If it turns out that you need more, you can always go back and try cleaning up whatever bottleneck there really is, whether that's a db connection or it's the fork/exec overhead, or something else. But to solve performance issues you may never have is really wasting resources: you.

  • Comment on Re^3: Simple (but robust) email server (receiver)

Replies are listed 'Best First'.
Re^4: Simple (but robust) email server (receiver)
by tirwhan (Abbot) on Dec 16, 2008 at 16:31 UTC
    First off, all the SMTP-handling code is in C. That's got to be less resource-intensive than perl.

    Not when I tested it. It's hard to accurately benchmark just the SMTP-talking part of an MTA, so I'll not claim those tests were perfect, but accepting multiple emails using Net::Server::Mail did not take significantly longer nor use more memory than doing so with a regular MTA.

    Second, it's already written, most bugs worked out, and someone else will fix any new bugs found.

    The same is true of Net::Server::Mail.

    Regarding the comparative resource-intensiveness of multiple perl processes versus a daemon, I think you're forgetting copy-on-write. You're right in that the OS will almost certainly keep modules in the disk cache if they are called multiple times by quickly-spawning processes. However, the OS still has to create a new process and address space, compile the Perl code and fill the new process memory with it. Do that several thousand times within a second and you will run out of memory even on beefy machines. With a perl daemon that forks off instances of itself, the OS uses COW for the process address space, which means that real memory consumption is far lower.

    Another point not mentioned yet is resource consumption limiting. If you have qmail spawning processes you have very little control over how many processes are created simultaneously. This can be especially fatal if your database connection is slow or some other resource is blocking, your OS will simply create processes too fast for the machine to handle and crash. This can be circumvented by using ulimit or PAM, but both of these have drawbacks, are not failsafe and far more complicated than simply setting max_servers in Net::Server::PreFork.


    All dogma is stupid.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://730673]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (3)
As of 2024-04-19 15:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found