sgifford has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to implement a standard protocol (RETS, to be specific) which defines certain responses as MIME multipart/parallel entities with CR/LF line endings for headers, with some MIME entities containing binary data. I'm trying to get CR/LF line endings out of MIME::Entity, but it's full of hardcoded \n characters, and so far I don't have a solution I'm happy with. Here's what I've tried:

Does anybody have any other suggestions? Thanks!

Replies are listed 'Best First'.
Re: Creating MIME entities with CRLF line endings
by Skeeve (Parson) on Mar 16, 2007 at 08:48 UTC

    Besides dk's suggestion it might be a good idea to contakt the author of MIME::Entity.

    Reading perldoc perlipc reveals this:

    Internet Line Terminators

    The Internet line terminator is "\015\012". Under ASCII variants of Unix, that could usually be written as "\r\n", but under other systems, "\r\n" might at times be "\015\015\012", "\012\012\015", or something completely different. The standards specify writing "\015\012" to be conformant (be strict in what you provide), but they also recommend accepting a lone "\012" on input (but be lenient in what you require).

    We haven't always been very good about that in the code in this manpage, but unless you're on a Mac, you'll probably be ok.

    So if MIME::Entity has hardcoded \n instead of \015\012, either there is some way to make sure \n spits out \015\012 and you haven't read the documentation fully (I haven't), or the author didn't know about the standard.


    s$$([},&%#}/&/]+}%&{})*;#$&&s&&$^X.($'^"%]=\&(|?*{%
    +.+=%;.#_}\&"^"-+%*).}%:##%}={~=~:.")&e&&s""`$''`"e
      Hi Skeve,

      Normally local mail messages are handled with local line endings, and the MTA takes care of line ending conversions if the message is sent ove rSMTP (which is why binary attachments are normally encoded as text. For example, if you send a message with sendmail or Net::SMTP, those programs will handle the line conversions for SMTP.

      So, it's not really a bug (or even that surprising) that MIME::Entity generates messages with local line endings, but it would be nice to be able to override it easily.

        Thanks for the explanation.

        s$$([},&%#}/&/]+}%&{})*;#$&&s&&$^X.($'^"%]=\&(|?*{%
        +.+=%;.#_}\&"^"-+%*).}%:##%}={~=~:.")&e&&s""`$''`"e
Re: Creating MIME entities with CRLF line endings
by dk (Chaplain) on Mar 16, 2007 at 08:34 UTC
    Did you consider to leave \n's of MIME:: as they are, collect whatever MIME:: outputs, and manually replace \n's to CRLFs? If you should adopt this method, you will have to maintain a clean division line, which scalars contain newlines encoded as CRLF, and which contain system-specific newlines. The conversion between these is trivial, s/\n/\xD\xA/g unless "\n" eq "\xD\xA" and vice versa.
      Thanks dk,

      The problem is that this will also change newlines in the binary parts. I could parse the message to figure out what parts are binary and ignore them, but then I will have gone and written a MIME parser, and I might as well have written a MIME generator.

Re: Creating MIME entities with CRLF line endings
by bart (Canon) on Mar 16, 2007 at 11:15 UTC
    It sounds to me like you should keep binmode on at all time, so no automatic conversion, and convert "\n" to "\015\012" only in the headers, by hand. AFAIK, constantly swapping between binmode and text mode for a handle doesn't work well.

    Note that

    substr($string, $start, $length) =~ s/\n/\015\012/g;
    should work fine. You only still have to locate the offsets/lengths of the substrings containing the headers.
      ... locate the offsets/lengths of the substrings containing the headers.

      That, and account for the fact that the substitution may change the length of a header, and thus the offsets to subsequent headers. It's often easiest to reverse() the list of header locations and work your way backwards.

      Anno

Re: Creating MIME entities with CRLF line endings
by rodion (Chaplain) on Mar 16, 2007 at 09:29 UTC
    Later Update: This is bad advice. See posts below by sgifford, bart and Anno.

    Try using the ":crlf" directive with binmode() on the file handle, as in

    binmode STDOUT, ':crlf'; $top->print(\*STDOUT); # or open MIME, '>', $fname; binmode MIME, ':crlf'; $top->print(\*MIME);
    I think this will do what you want.

    Updated:

    Thinking about it a little further, since MIME::Entity's print method takes a file handle as a parameter, it makes sense that it would leave control of line termination for the output file to the module that manages the output file handle, while using "\n" as the conventional line terminator within code. The binmode() function is Perl's way of controlling line termination on a file handle.

      This converts binary parts too, which corrupts them if they contain the byte 0x0a.