Yes, about all you can do is check if a line is short some fields, and if it is and the next line is also short, put them together. But this is further complicated by the possibility that the newlines could be inside a quoted field, in which case the CSV parsing will choke and you have to figure out a much more complicated way of counting fields. And the file could theoretically not have a header, or the header might not have names for all the fields, in which case figuring out how many fields the file -should- have is difficult as well. And what if the field with the newline is at the very end of the record and unquoted? It'll look like that record is fine and the problem starts on the next record.

This is one of those problems I wish I had an easy answer for as well. I've run into it a number of times while doing data imports for one of my clients.

In reply to Re: Handling an embedded newline in an unquoted CSV field by TJPride
in thread Handling an embedded newline in an unquoted CSV field by olivierp

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.