Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Filtering rows with Parse::CSV

by Anonymous Monk
on Mar 27, 2017 at 18:02 UTC ( [id://1186108]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello, and thanks in advance for your help.

I'm attempting to get all rows from a CSV file where a specific column is not null. Unfortunately, I'm only getting 1 row back.

For example, I'd like only rows 5, 7, and 9 to be printed.

row 1-4 : myColumn is null
row 5 : myColumn = 'ABC'
row 6 : myColumn is null
row 7 : myColumn = 'CDE'
row 8 : myColumn is null
row 9 : myColumn is 'EFG'

My expected results are:
ABC
CDE
EFG

Instead, my results are:
ABC

Here's the code I am using:

my $parser = Parse::CSV->new( file => $CSV_File, sep_char => ',', names => 1, filter => sub { $_->{myColumn} ? $_ : undef } ); while ( my $row = $parser->fetch ) { my $myColumn_value = $row->{'myColumn'}; print ("$myColumn_value"); }

Replies are listed 'Best First'.
Re: Filtering rows with Parse::CSV
by AppleFritter (Vicar) on Mar 27, 2017 at 18:07 UTC

    Off the top of my head and untested:

    while (my $row = $parser->fetch()) { my $myColumn_value = $row->{'myColumn'}; print $myColumn_value unless($myColumn_value eq ""); }

    if (and unless) can act as statement modifiers in Perl, so this will cause the print to be executed only if $myColumn_value isn't equal to the empty string.

    You will get a warning here if it's undefined, BTW. If that's an issue, use

    print $myColumn_value unless(($myColumn_value // "") eq "");

    instead.

Re: Filtering rows with Parse::CSV
by toolic (Bishop) on Mar 27, 2017 at 18:14 UTC
    I don't see any commas in your input. Does your input actually have "row 1-4:"? Post the 1st 9 lines of your input file, exactly as they appear in the file. Use "code" tags.
      Thanks for your help. That was it! The first 2 columns had commas within the data. I removed the first two columns, and it ran as expected.

      UPDATE

      I may have spoken too soon. As soon as I put the columns back (along with several other columns I removed), I ran into issues again. The filter is working as expected, however, the while loop exits after about 240 rows. I'm assuming it's due to the size of the file (155k KB)... Is there a size limit to the Parse::CSV module?

        the while loop exits after about 240 rows. I'm assuming it's due to the size of the file
        I doubt if your problem is due to the size of the file - Parse::CSV is designed specifically to handle large files, and only reads one line at a time.

        You could try adding this line after your loop:

        print $parser->errstr;
        I suspect, though, that the problem has something to do with the structure of your data - does it change in any way around row 240? You mention that some of the columns have commas within - maybe you could post some of your input data here (inside <code></code> tags).

        What module version do you have, add this line to see

        print $Parse::CSV::VERSION;
        poj
Re: Filtering rows with Parse::CSV
by Tetris78 (Initiate) on Mar 27, 2017 at 18:08 UTC
    Thanks again. I just registered so that I am not anonymous.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1186108]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (6)
As of 2024-03-29 13:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found