I am having a regexp problem which is a couple of common regexp problems with a twist. My regexp skills are not up to snuff so I am sort of stuck.

I am working in a csv file, and ultimately:
- I need to parse it, and change the delimeter
- keep the commas that were within the strings
- remove the double quotes signifying strings
for example:

1,2,3,4,"foo, bar, and bob", "example",5,6,"plip,plop"

this would end up as:
1~2~3~4~foo,bar, and bob~example~5~6~plip,plop

Handling the commas is whats stopping me...

I was trying to do it seperately from all the other goals and was using:
$_ =~ s/,\s*(\".*?),(.*?\")/$1-=O=-$2/g;
This piece of code skips by multiple commas which get matched in $2.

The files I have to manipulate are -=huge=- so efficiency is important. unfortunately, the more I goof with this, the less efficient it becomes :P I know there are some modules out there which can help with parsing, but I was unfamiliar as to whether it would give me the speed boost that I need.

Thanks you all for listening to my rambling.
Below is the file in case what I said still doesn't make sense.

use strict;
my $data_file = $ARGV[0]; #input file
my $old_delim = $ARGV[1]; # old delimeter
my $output_file = $ARGV[2]; # out put file
my $new_delim = $ARGV[3]; # new delimeter
my $count =0;

print "\n$0 Started.....\n\n";
print " Input file name: $data_file\n";
print " File delimiter $old_delim\n";
print " Output file name: $output_file\n";
print " File delimiter $new_delim\n";

open(INFILE, "$data_file") || die "INPUT ERROR: $!\n"; #open file or k
+ill the script and return the error
die "INPUT ERROR: input file is empty\n" if( -s $data_file < 1); # if 
+the file is empty, kill the script and print out an error message

open(OUTFILE, ">$output_file") || die "OUTPUT ERROR: $!\n"; 


while(<INFILE>){
    $_ =~ s/,\s*(\".*?), (.*?)\"/$1-=O=-$2/g;
    $_ =~ tr/\\\"//d;
    my @words = split($old_delim,$_);
    my $newline =  join($new_delim,@words);
    $newline =~ s/-=O=-/,/g;
    print OUTFILE $newline;
    $count++;
}



print "line count: $count\n";
print ("\n\nCompleted.\n\n");

[download]

In reply to regexp problems by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.