Filtering rows with Parse::CSV

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello, and thanks in advance for your help.

I'm attempting to get all rows from a CSV file where a specific column is not null. Unfortunately, I'm only getting 1 row back.

For example, I'd like only rows 5, 7, and 9 to be printed.

row 1-4 : myColumn is null
row 5 : myColumn = 'ABC'
row 6 : myColumn is null
row 7 : myColumn = 'CDE'
row 8 : myColumn is null
row 9 : myColumn is 'EFG'

My expected results are:
ABC
CDE
EFG

Instead, my results are:
ABC

Here's the code I am using:

my $parser = Parse::CSV->new(
    file       => $CSV_File,
    sep_char   => ',',
    names      => 1,
    filter     => sub { $_->{myColumn} ? $_ : undef }
);

while ( my $row = $parser->fetch ) { 
     my $myColumn_value = $row->{'myColumn'};
     print ("$myColumn_value");
}
[download]

Comment on Filtering rows with Parse::CSV Download Code

Replies are listed 'Best First'.
Re: Filtering rows with Parse::CSV by AppleFritter (Vicar) on Mar 27, 2017 at 18:07 UTC
Off the top of my head and untested: `while (my $row = $parser->fetch()) { my $myColumn_value = $row->{'myColumn'}; print $myColumn_value unless($myColumn_value eq ""); }` [download] `if` (and `unless`) can act as statement modifiers in Perl, so this will cause the `print` to be executed only if `$myColumn_value` isn't equal to the empty string. You will get a warning here if it's undefined, BTW. If that's an issue, use `print $myColumn_value unless(($myColumn_value // "") eq "");` [download] instead.	[reply] [d/l] [select]
Re: Filtering rows with Parse::CSV by toolic (Bishop) on Mar 27, 2017 at 18:14 UTC
I don't see any commas in your input. Does your input actually have "row 1-4:"? Post the 1st 9 lines of your input file, exactly as they appear in the file. Use "code" tags.	[reply]
Re^2: Filtering rows with Parse::CSV by Tetris78 (Initiate) on Mar 27, 2017 at 18:44 UTC
Thanks for your help. That was it! The first 2 columns had commas within the data. I removed the first two columns, and it ran as expected. UPDATE I may have spoken too soon. As soon as I put the columns back (along with several other columns I removed), I ran into issues again. The filter is working as expected, however, the while loop exits after about 240 rows. I'm assuming it's due to the size of the file (155k KB)... Is there a size limit to the Parse::CSV module?	[reply]
Re^3: Filtering rows with Parse::CSV by tangent (Parson) on Mar 27, 2017 at 21:24 UTC
the while loop exits after about 240 rows. I'm assuming it's due to the size of the file I doubt if your problem is due to the size of the file - Parse::CSV is designed specifically to handle large files, and only reads one line at a time. You could try adding this line after your loop: `print $parser->errstr;` [download] I suspect, though, that the problem has something to do with the structure of your data - does it change in any way around row 240? You mention that some of the columns have commas within - maybe you could post some of your input data here (inside <code></code> tags).	[reply] [d/l]
Re^3: Filtering rows with Parse::CSV by poj (Abbot) on Mar 27, 2017 at 21:31 UTC
What module version do you have, add this line to see `print $Parse::CSV::VERSION;` [download] poj	[reply] [d/l]
Re^4: Filtering rows with Parse::CSV by Tetris78 (Initiate) on Mar 29, 2017 at 14:47 UTC
Re: Filtering rows with Parse::CSV by Tetris78 (Initiate) on Mar 27, 2017 at 18:08 UTC
Thanks again. I just registered so that I am not anonymous.	[reply]