When a line ends with a comma, you're able to join it with the following line, by removing the line-break(s) after the comma. (But your regex does it wrong: it won't apply at all on LF-style data, it won't handle extra blank lines properly for CRLF-style data, and it removes the comma, which should be kept.)

When a line begins with a comma, you want to join it to the previous line, but the previous line has already been processed and written to output, so it's too late to fix that.

So, don't do it one line at a time - process the whole file as a single string:

perl -e '$/=undef; $_=<>; s/,[\r\n]+/,/g; s/[\r\n]+,/,/g; print' infil
+e > infile.fxd
[download]

(Note that I'm redirecting output to a different file, rather than replacing the original - that makes it easy to "try, try again" for cases like this, where you seldom get it right the first time. Once you get it right, then you can rename the output to replace the input.)

(Update: P.S.: Welcome to the Monastery!)

(Updated again to add remarks about LF vs. CRLF data)

This works perfectly. But it deletes the very first line. How to prevent this?

… But it deletes the very first line. How to prevent this?

Can you post an example data file to demonstrate this problem? (For example, just copy the first few lines from the "real" data file in question and put them in a separate file. Then run the command line to produce a modified version of that. If the first line of input is missing from the output, please post the input, the command line you actually used, and the output.

I suspect that one (or more) of the following could be happening:

• You aren't using the exact code that I posted.
• What you think should be "the first line" of your input file is not actually in that file to begin with (that is, it was missing before you ran the script).
• There's something goofy going on with carriage-return characters in your data, and the first line really is there, but you might not be "seeing" it because maybe it ends with just CR instead of CRLF, which might cause the 2nd line to be printed "on top of" the first one in your display.

I have no way to answer your question without knowing more about your data.

I think graff summarized your problem quite well. "Fixing" CSV with regular expressions is fragile. It is possible for some very well-defined situation but likely to break in another.

What (usually) works better is to use a well-defined parser like Text::CSV_XS and remember what a sane state is: either scan each line knowing how many fields should be in it or dynamically remember what the first sane line told you. If a line gives you a parsing error or the parsed line does not contain enough fields, back up and join the next line. You can take the beforementioned parser-xs.pl as a framework to start from.

In your case, that could end up with something like (untested) this:

use strict;
use warnings;

use Text::CSV_XS;

my $csv = Text::CSV_XS->new ({ binary             => 1,
                               blank_is_undef     => 1,
                               eol                => $/,
                               });
my $csa = Text::CSV_XS->new ({ binary             => 1,
                               allow_loose_quotes => 1,
                               blank_is_undef     => 1,
                               escape_char        => undef,
                               });

my $file = @ARGV ? shift : "test.csv";
open my $fh, "<", $file or die "$file: $!\n";

my %err_eol = map { $_ => 1 } 2010, 2027, 2031, 2032;

print STDERR "Reading $file with Text::CSV_XS $Text::CSV_XS::VERSION\n
+";
my $nf = 0;
my @pv;
while (1) {
    my $row = $csv->getline ($fh);

    if ($row) {
        if (@pv) {
            # previous line ended with embedded newline.
            $pv[-1] .= "\n" . shift @$row;
            unshift @$row, @pv;
            @pv = ();
            }
        $nf ||= @$row;
        if (@$row < $nf) {
            @pv = @$row;
            redo;
            }
        }
    else {      # Parsing failed

        # Could be end of file
        $csv->eof and last;

        # Diagnose and show what was wrong
        my @diag = $csv->error_diag;
        print STDERR "$file line $./$diag[2] - $diag[0] - $diag[1]\n";
        my $ep  = $diag[2] - 1; # diag[2] is 1-based
        my $ein = $csv->error_input;    # The line scanned so far
        my $err = $ein . "         ";
        substr $err, $ep + 1, 0, "*";   # Bad character marked between
+ **
        substr $err, $ep,     0, "*";
        ($err = substr $err, $ep - 5, 12) =~ s/ +$//;
        print STDERR "    |$err|\n";

        REPARSE: {      # Now retry with allowed options
            if ($csa->parse ($ein)) {
                print STDERR "Accepted in allow mode ...\n";
                $row = [ $csa->fields ];
                }
            else {      # Still fails
                my @diag = $csa->error_diag;
                if (exists $err_eol{$diag[0]}) { # \r or \n inside fie
+ld
                    print STDERR "  Extending line with next chunk\n";
                    $ein .= scalar <$fh>;
                    goto REPARSE;
                    }

                print STDERR "  Also could not parse it in allow mode\
+n";
                print STDERR "  $./$diag[2] - $diag[0] - $diag[1]\n";
                print STDERR "  Line skipped\n";
                next;
                }
            }
        }

    # Data was fine, print data properly quoted
    $csv->print (*STDOUT, $row);
    }
[download]

:-(