blank lines in email

jason_s has asked for the wisdom of the Perl Monks concerning the following question:

Hi! I've begun using Email::Stuff in my scripts lately. I'm having a problem, where it's putting extra blank lines after newlines. I don't see this behavior when sending directly through sendmail. Here's my test program:

#!/usr/bin/perl
use warnings;
use strict;
use Email::Stuff;

my $body="line1\nline2\n";
my $to='jason@example.com';

#use email::stuff
Email::Stuff->to($to)
            ->subject("hello from Email::Stuff")
            ->text_body($body)            
            ->send;

#use sendmail
open ( MAILER, "|/usr/lib/sendmail -t" );
print MAILER "To: $to\n";
print MAILER "Subject: hello from sendmail\n";
print MAILER "$body";

close MAILER;
[download]

When I run it, I get two mails (headers scrubbed): The mail sent using Mail::Stuff has a blank line in it:

Date: Fri, 15 Jan 2010 15:12:38 -0800
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: quoted-printable
To: jason@########.com
Subject: hello from Email::Stuff
Message-Id: <20100115231238.CD5CE2DBBBA@evo>
From: jason@evo.u.lg.########.com (Jason Speck)

line1

line2

The mail sent via sendmail does not have a blank line in it.

To: jason@######.com
Subject: test using sendmail
Message-Id: <20100115231238.D9DBC81EACA@evo>
Date: Fri, 15 Jan 2010 15:12:38 -0800 (PST)
From: jason@evo.u.lg.######.com (Jason Speck)

line1
line2

I suspect that Email::Stuff is doing what it's supposed to, but I'm just not understanding something about how mail works. I'd like to understand (1), where the blank lines after newlines are coming from, and (2) how to get rid of them so that I can get the formatting that I want.

Comment on blank lines in email Download Code

Replies are listed 'Best First'.
Re: blank lines in email by sflitman (Hermit) on Jan 16, 2010 at 05:48 UTC
Dump the bytes of the mail, I'll bet the module is expanding \n to \r\n or something like that. You can use the as_string function instead of send to look at the full MIME message before it is sent. Completely other idea: could the sendmail on your system be configured oddly? Or is it being called with different switches than -t from Email::Stuff? HTH, SSF	[reply]
Re^2: blank lines in email by jason_s (Acolyte) on Jan 18, 2010 at 22:24 UTC
Thanks for the tip. I used as_string, and it shows that the module is indeed expanding the \n to \r\n. I'm using postfix as my MTA. It sounds like postfix on a Unix system will blindly change any \n to \r\n, trying to helpfully give you the right CRLF line terminator...The extra carriage return is what's causing my problem. According to the exim documentation, exim's MTA handles this more intelligently, which is probably why gmargo couldn't duplicate this. This SEEMS to me like a bug in Email::Stuff or one of its dependencies. My understanding is that the Unix convention is for the MUA to use the local line terminator (a bare newline), and that the MTA is responsible for massaging this correctly for internet delivery...	[reply]
Re^3: blank lines in email by gmargo (Hermit) on Jan 18, 2010 at 23:18 UTC
Can you please post exactly what Linux distribution and version you are using? And versions of the relevant modules? I'll see if I can duplicate your setup in a virtual machine.	[reply]
Re^4: blank lines in email by jason_s (Acolyte) on Jan 19, 2010 at 02:05 UTC
Re^3: blank lines in email by jason_s (Acolyte) on Jan 18, 2010 at 23:15 UTC
I tried the same test using Email::Sender::Simple and MIME::Lite. Both of those modules did the right thing. I guess I'll just switch to using one of them.	[reply]
Re: blank lines in email by holli (Abbot) on Jan 16, 2010 at 00:01 UTC
What's the value of "\n" on your system? holli You can lead your users to water, but alas, you cannot drown them.	[reply] [d/l]
Re^2: blank lines in email by jason_s (Acolyte) on Jan 16, 2010 at 00:05 UTC
I'm on Linux, so it's linefeed.	[reply]
Re: blank lines in email by gmargo (Hermit) on Jan 16, 2010 at 15:01 UTC
I duplicated your test but got the expected result with no extra blank line. I'm using Ubuntu 8.04 Hardy. For this test I used the following newer version modules; all others are the Hardy default. I'm also using `exim4` for the MTA instead of `postfix`, but I don't think that should matter. Email-Date-Format-1.002 Email-MIME-1.902 Email-MIME-Encodings-1.313 Email-Simple-2.100 Email-Stuff-2.101	[reply] [d/l] [select]
Re: blank lines in email by doug (Pilgrim) on Jan 17, 2010 at 17:11 UTC
SMTP always uses \r\n end of lines as it is required in RFC 822. When I cared about such things, my solution was to `s/[\00-\32]+$//` every input line. This gets rid of trailing white space, all EOLN characters and generally works well in mixed environments. It might not be very unicode friendly, but that was never an issue for me. When you print it out simply `print map { "$_\n" } @stripped_data;` and you always have the local EOLN regardless of what your data was. - doug	[reply] [d/l] [select]