in reply to search/grep perl/*nix

Did you mean something like this?

Data like 1nickt but with dups.

fred,barney,wilma,betty foo,bar,baz,qux foo,bar,baz,qux fred,barney,wilma,betty fred,barney,wilma,betty apple,orange,banana,pear apple,orange,banana,pear foo,bar,baz,qux fred,barney,wilma,betty apple,orange,banana,pear
#!/usr/bin/env perl use strict; use warnings; use feature qw (say); use Data::Dump; use Iterator::Simple qw(iter); my $file = q(1204245.csv); my %column; open my $fh, q(<), $file or die $!; while (<$fh>) { chomp; $column{ ( split /,/ )[2] } = undef; } close $fh; dd \%column; # what ever.. say for sort { $b cmp $a } keys %column; say q(--); # or my $iterator = iter( IO::File->new($file) ); while (<$iterator>) { chomp; $column{ ( split /,/ )[2] } = undef; } dd \%column; say for sort { $a cmp $b } keys %column; __END__ karls-mac-mini:monks karl$ ./1204245.pl { banana => undef, baz => undef, wilma => undef } wilma baz banana -- { banana => undef, baz => undef, wilma => undef } banana baz wilma
"...dataset get significantly larger..."

Perhaps the solution with Iterator::Simple performs better. But I guess - and your mileage may vary. See also Benchmark.

Best regards, Karl

«The Crux of the Biscuit is the Apostrophe»

perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help