SpacemanSpiff has asked for the wisdom of the Perl Monks concerning the following question:

Good evening, Monks! Obviously there's no shortage of modules out there to deal with CSV files -I'd be crazy to suggest otherwise. So that leaves it to either my inexperience with the language or the lack of relavent examples leaving me rather high and dry. I found a few modules on CPAN that I'd like to try, but pretty much all I can see is examples of how to fill an array and pump it into a new CSV. Not much on searching and modifying existing data. See, I have a memberlist that I need to search to see if members exist. If they do, I need to increment a counter for that user. If not, I need to add them and start a new counter. Here's an example of the format:

Username,Email,Counter,JoinDate

Now again, being new, I'm learning by example here. Can anyone point me in the right direction and give a good firm kick? Class::CSV looked as though it had promise, as did Text::CSV_XS. I was even looking at DBD::RAM for a while, but I've come to the conclusion that it's not available for my current build of ActivePerl. Since the file is only about 3500 rows by 7 or so columns, I don't think reading it into RAM is a problem. Then again, it's not entirely necessary either. While I don't want this to take years to get through, I don't need to set any speed records either. Reading and writing the existing file should suffice.

  • Comment on CSV files: Searching, manipulating data, appending and saving.

Replies are listed 'Best First'.
Re: CSV files: Searching, manipulating data, appending and saving.
by GrandFather (Saint) on Oct 12, 2005 at 04:07 UTC

    Have a look at the various replys to this node. I give a CSV and DBI solution to the OP's problem that may interest you.


    Perl is Huffman encoded by design.
      Looks like exactly what I'm trying to do. The problem is, I'm having trouble adapting it. To be honest, I'm trying to follow what the commands are doing. It appears if I create the file it's trying to manipulate and run it as is, I just get errors. If possible, could you do a play by play? The POD starts making sense, then in their example I see this:

      $sth = $dbh->prepare("SELECT foo, bar FROM table WHERE baz=?"); $sth->execute( $baz ); while ( @row = $sth->fetchrow_array ) { print "@row\n"; }
      *puts on dunce cap* Where did $baz come from? Was it being assigned in the previous line after the WHERE statement?

      **I should mention that I'm looking at the DBI example you gave, although that was probably already obvious.**

      Thanks again!

        Ok, it would appear that I've got it reading the CSV finally. I'll work on the manipulating and saving bit in the morning.
        Ok, I've got the command returning the contents of the whole CSV file. The question now is how do I locate, one specific row and affect it's data? Here's what I have:

        #!/usr/bin/perl -w use strict; use DBI; use File::Basename; my $dir = '.'; my $file = 'memberlist.csv'; my $table = (fileparse($file,'.csv'))[0]; my $cols = [qw(Username Email Posts EmailSet Group Unknown Date Times +tamp Password MD5)]; my $sep = ','; my $dbh = DBI->connect( "DBI:CSV:f_dir=$dir;csv_eol=\n;csv_sep_char=$sep;", {RaiseError=>1},); $dbh->{csv_tables}->{$table} = { file => $file, col_names => $cols,}; my $sth = $dbh->prepare("SELECT Username, Email, Posts, Timestamp +FROM memberlist"); $sth->execute() or die "Cannot execute: " . $sth->errstr(); while ( ( my $Username, my $Email, my $Posts, my $Timestamp) = $st +h->fetchrow_array) { print "$Username, $Email, $Posts, $Timestamp\n"; } $sth->finish();
        Now I tried an 'if' command within the 'while' statement to match up an email, then increment it's '$post' variable once it hit a specified address (a test of the function I need to do) and all it did was increment everyone's post count. At this point, I wonder if I'm on the right track. Should I even be pulling the value into the script, or should I be looking at an 'update' command of some sort? I've read elsewhere that someone just read each line into an array and went from there, but that sounds like a lot of overhead to read it each time. Likewise, I'd be concerned about holding the whole table in memory in the event of an error during the process.
Re: CSV files: Searching, manipulating data, appending and saving.
by jZed (Prior) on Oct 12, 2005 at 05:14 UTC
    DBD::CSV is designed for exatly the kind of thing you're talking about. It works pretty much like DBD::RAM so if you found that usable, DBD::CSV will be easy to pick up.
      Honestly, I've had mild trouble with all of them, but I'm still swinging away at it. This is next on the list. Thanks for the reply!