"...not allowed to use any modules?"

Further: What did you try? How did it fail?

We like to know the answers to these questions...because the Monastery's mission is to help you learn; not to do your homework.

Read perlre. If you're in a rush read perlrequick. And if you find the learning curve too steep, take a step back and start with perlretut

Note, this isn't perfect. It assumes your end of line character is a linefeed. It also doesn't handle linefeeds in the 4th field. But then in the sample provided, it wasn't significant.

use strict;
use warnings;

my $data;
localscope: {
    local $/;
    $data = <DATA>;

    my $i=1;
    while ($data =~ /\G((?:(?:[^,]*),){3}(?:[^\n]*\n))/g) {
    my $entry = $1;
    print "$i $entry";
    $i++;
    }
}

1;
__DATA__
"123", "DEF123","this 
is
 test","C:\Abhinav\test.jpg"
 "456", "DEF456","this is test","C:\Matt\test.jpg"
 "726", "DEF726","this is test","C:\Matt\test.jpg"
[download]

I would also recommend Jeffrey Friedl's Mastering Regular Expressions book. I haven't read all those pods to the end yet so perhaps they cover all that's needed, but I can say his book helped me a great deal and touches specifically on some of the issues here (assuming you can't be convinced to use a module). For instance, I didn't properly understand how to match across newlines (or, sadly, even that newline is one of the things \s matches -- yikes) or what the /m and /s (and /ms) qualifiers do exactly until I read his book. The information was probably there in the perl pods but my eye must have glazed over it or struggled with the wording.

It also deals with how to match within quotes (and how to do so efficiently), to the point where Damian Conway's Perl 6 Exegesis 5 document even refers to a certain kind of regex as being "Friedl style".

If the latest edition is too long for you, the one I read (recently) was the 1st edition, and I can say that it's still valuable, even if it does leave out some newer Perl regex features. Just in case, I read parts of the chapters on dfas and nfas in a newer edition from the library and I think perhaps the coverage there was flushed out and improved some (have to say the car analogy is the single thing I dislike about the book -- maybe I'd be less bothered if it was a bicycle analogy, I dunno), but the bulk of the size increase seemed to come from covering more regex flavours from more languages (ones that I don't personally care about as much as the ones in the 1st edition). Someone correct me if this is a poor impression to put out into the world.

Hi!

• Advice #1: use a module to parse such files, e.g.Text::CSV (csv = comma seperated values). Don't use a regex for it.
• Advice #2: check the documentation of that module how to handle the newlines. E.g. Text::CSV advises you to use binmode.

HTH, Rata

merging csv files into a third file preserving column & row

If we are to take you at your word that this is not homework, then the stricture that you are not allowed to use any modules makes no sense.   The purpose, and the power, of Perl, is to leverage what someone else has already done.   You have received excellent suggestions, and you should, I think, go to your manager and explain to him or her that the most expeditious way to solve the problem is to use one of these packages.   (When you want to install a new door in your house, you do not begin by cutting down a tree with an axe ...)

Remember that packages can be installed locally, even on a per-application basis, without disrupting anything else that is installed.   This is discussed in topics that include non-root user in their titles here, because it is an issue that is constantly dealt with when installing software on a shared-hosting web server.

I can relate to the OP. My employer is actually afraid of Perl for "security reasons" and generally won't install any modules. Silly, given that Perl's one of the best tools out there in terms of security. Not all the neanderthals are dead...

I could almost understand that if modules required root access and a sysadmin to install (if only because I'm a sysadmin, and I don't want to be bothered for stuff like that), but with things like local::lib and even just running cpan as a non-privileged user, it's hard to justify or make sense of it.

Christopher Cashell

You should use Text::CSV!

use strict; use warnings;
local $/='';
my @data = split /"\s*"/, <DATA>;
print join "\n\n\n\n", @data;

__DATA__
"123", "DEF123","this 
is
 test","C:\Abhinav\test.jpg"
 "456", "DEF456","this is test","C:\Matt\test.jpg"
 "726", "DEF726","this is test","C:\Matt\test.jpg"
[download]

*Update* : Oh yes .. I forgot to mention.. i'm not allowed to use any modules..

Parsing CSV isn't your real problem; this is your real problem.

CSV is one of those things that seems simple and easy to handle at first glance. But, once you start digging in and coming across corner-cases, you realized it can be very tricky to get right. You mention the embedded newlines, but what about embedded commas (commas inside quoted fields)? Are all fields quoted? Can you guarantee that? What about escaped quotes inside of quoted fields?

Unless this is a homework assignment, or you have strange desire to reinvent the wheel (in which case you should be reviewing the code in existing CSV modules), you would be better off copying and pasting Text::CSV into your program than you would be in trying to recreate it.

Funny, that when people post saying "I'm not allowed to use modules", I almost never hear them say why.

I was thinking the exact same thing. I'm not going to say that there are no valid reasons or situations where modules aren't allowed, but it's very hard to think of one that doesn't involve, "because I'm supposed to write this functionality myself as homework".

worst-case scenario, you can almost always bundle the modules into your own application, or utilize modern tools like App::FatPacker to do the work for you.

Programming in Perl without CPAN is like going into a gun fight without bullets.

Christopher Cashell

$out_file = "testout.txt"; #this is the file to be edited


open (OUT, "<$out_file") or die "Can't open $out_file: $!\n";

 $/ = "\n";

  while ($line = <OUT>) {
    
   my $first = (split /,/, $line)[0];
   $first =~ tr/"//d;
   $first =~ s/^\s+//;
 
 .......
  }

close OUT;
[download]

Did something similar years ago --- so...

#!\user\bin\perl -w

while (@a_Field=GetCSV(@a_Field)) {
  print join(' | ',@a_Field)."\n\n";
   };

  sub GetCSV{
    my($s_Line,$s_Text);
    while ($s_Line=<DATA>) {
      #print "'$s_Line'\n";
      my(@a_Field);
      $s_Text.=$s_Line;
      while ($s_Text =~ m{([^",]*)([,\n])|"((?:[^"]|"")*)"([,\n])}gs) 
+{
        unless (substr($`,-1,1) eq '"') {
          if (defined $1) {
            push(@a_Field,$1);
            if ($2 eq "\n") {
              return @a_Field;
               };
             } else {
            my($s_Field);
            ($s_Field=$3)=~s/""/"/g;
            push(@a_Field,$s_Field);
            if ($4 eq "\n") {
              return @a_Field;
               };
             };
           };
         };
       };
    return ();
     };

__DATA__
10915,"S","Phil","ing Valley Middle School",$0.00,0,,0
10916,"Tr","Ny",,,,,,"999-999-9999",,,,"715000 works at Re-Max
Wishing for housecoat with dogs, no luck",$0.00,0,,0
10917,"Ro","Ox","3677 As Dr","W","BC","V4T 2W5",,"1111111111",,,,,$0.0
+0,0,,0
10918,"Sa","Fri",,"K","BC",,,"2222222222",,,,,$0.00,0,,0
10355,"Val","Woj",,,,,,"3333333333",,,,"Solutions",$0.00,0,,0
10356,"Ter","Bes",,,,,,"1211211212",,,,,$0.00,0,,0
10357,"Phi","Har",,,,,,"1231231234",,,,"6 Woodcroft Ave
St Catns
x1x1x1
999-999-3203
check the address to see if it is still her's.",$0.00,0,,0
10358,"Ra","Gak",,"Kel",,,,"3453453456",,,,,$0.00,0,,0
10359,"J","Ru",,"V","BC",,,"7777777777",,,,"st ing to tell J to come a
+gain…
(555) 899-9999",$0.00,0,,0
10360,"Li","Sa",,"Win",,,,"4444444444",,,,"LDr Claremont",$0.00,0,,0
10361,"Ke","Ta",,"K","BC",,,"5555555555",,,,,$0.00,0,,0
10362,"Kat","son",,"V","BC",,,"6666666666",,,,,$0.00,0,,0
[download]

If the first record -- in fact -- is the same as the first record -- as posted -- how do you transform it to the sample output you offer. IOW, how does:

"123", "DEF123","this 
 is
  test","C:\Abhinav\test.jpg"</i>"  # Abhinav
[download]

become:

"1. "123", "DEF123","this 
 is
  test","C:\Matt\test.jpg"  # Matt???
[download]

Were you merely careless about posting your request for a way to resolve your problem. Carelessness like that often wastes the time of those good enough to try to help you!

And, not just BTW, I'm not persuaded by your claim that this is not homework. I'm not sure it is, either, but the 'no modules' statement -- in the face of advise (to use a CSV approach) from those far wiser about Perl than thee -- is pretty hard to accept as anything other than nonsense.

Also note: your comment in Re: Regex Split and Formatting that you "failed miserably" is not a precise statement of a problem... other than failure to persevere in the face of discouragement. The code accompanying that generality might serve to offer you some enlightenment, however, were you to add a few strategic print statements to see what you're really getting into $line and $first.

G'day reaper9187,

This does what you want:

$ perl -Mstrict -Mwarnings -e '
    my $cols = 4;
    my $re = qr{("(?:[^"\\]++|\\.)*+"\s*,*\s*)}m;
    my $file = do { local $/; <> };
    my @params = $file =~ /$re/g;
    print "$_. ", splice(@params, 0, $cols), "\n" for 1 .. @params / $
+cols;
'
 "123", "DEF123","this 
is
 test","C:\Abhinav\test.jpg"
 "456", "DEF456","this is test","C:\Matt\test.jpg"
 "726", "DEF726","this is test","C:\Matt\test.jpg"
1. "123", "DEF123","this 
is
 test","C:\Abhinav\test.jpg"
 
2. "456", "DEF456","this is test","C:\Matt\test.jpg"
 
3. "726", "DEF726","this is test","C:\Matt\test.jpg"
[download]

For the regexp, I've made a very minor addition to what's provided in perlre (search for "match a double-quoted string"). I also used splice to access groups of, what you're calling, parameters.

In future, can I suggest you state something like this in your OP: "I'm aware of Whatever::Module but I want to do this using other_thing because reason_why.". I think that might have saved a lot of questions/answers/updates/etc.