Email::MIME generating emails w/ large attachments kills Apache

trillich has asked for the wisdom of the Perl Monks concerning the following question:

We're trying to use Catalyst to let a web-user fill out a form, click 'submit' and then an email gets generated with relevant attachments.

But when the files get to be just a couple dozen K in size, Apache grinds to a halt and dies!

Is there an Email::Buffer library or approach that allows us to hand off file-path-names or maybe open filehandles so that the binary data can be encoded on the fly instead of doing full slurps on all the files? Or is there another conceptual approach to use altogether...?

Comment on Email::MIME generating emails w/ large attachments kills Apache

Replies are listed 'Best First'.
Re: Email::MIME generating emails w/ large attachments kills Apache by Eliya (Vicar) on Dec 22, 2011 at 01:01 UTC
Apache grinds to a halt and dies! What exactly does this mean? Does it run out of memory, or use too much CPU or I/O to finish the requests in time, or segfault (wrt the "dies"), or something else? If it's CPU, how many requests (on average) do you need to handle per second? I'm not asking "just because". Rather, I think a better understanding of what exactly the problem is, is essential for choosing an appropriate strategy to cope with it...	[reply]
Re: Email::MIME generating emails w/ large attachments kills Apache by TJPride (Pilgrim) on Dec 22, 2011 at 03:05 UTC
Need to find out what the problem is before it can be fixed, obviously. There is no practical reason why attachments that small should cause Apache, your email software, Perl, etc. to die. A good start would be to post your email generation code so we have something to look at. Also check your error logs and find out what the specific error is and post that as well.	[reply]
Re: Email::MIME generating emails w/ large attachments kills Apache by locked_user sundialsvc4 (Abbot) on Dec 22, 2011 at 10:09 UTC
You have an unrelated bug somewhere ... files of these sizes are inconsequential, and nothing should cause Apache to “grind to a halt and die.” Therefore, I cordially suggest that you are chasing a red herring right now. You have a bug somewhere in your code. Set up a simple way to invoke “the guts of the thing” directly from the command-line with some existing disk file (representing the already successfully-received upload content) as its input. Use this to find the hole in the algorithm which is most likely an endless (or near-endless) loop. Factor both the Web and Catalyst out of your investigation, as it is sufficiently unreasonable that either of these are the source of your problem. (In my experience, it’s probably going to turn out to be “a friggin’ line of code or three, obvious when you see it,” and you’re gonna slap your forehead and say, “Doh!!”)
Re: Email::MIME generating emails w/ large attachments kills Apache by trillich (Sexton) on Dec 22, 2011 at 19:39 UTC
Solution: Switched to using Mail::Sender -- it opens a channel directly to the outgoing mail server, so there's no pre-building of anything in RAM, it sends the encoded data immediately. Sweet! Not sure what the original crash was. Apologies, I should definitely have done some more diligence to be more specific... And I also would bet it was a forehead-slapper. But with a complete revamp of our View::Email to rely on Mail::Sender, it's a non issue now -- all is sparkles and sunshine again.	[reply]