Paulster2 has asked for the wisdom of the Perl Monks concerning the following question:

Question to ye of Noble Status:

Is there a module for sorting and getting unique entries in a file? I suppose that I could write something into my code, and maybe this is PERL101 stuff, I guess I'm just lazy, like most guys I know.

Thanks. Any help is appreciated

UPDATE: I withdraw my question on the grounds of utter stupidity.

Paulster2

Replies are listed 'Best First'.
Re: Sorting Unique File Entries
by l3nz (Friar) on Nov 24, 2003 at 13:10 UTC
    This is a very simple pure Perl solution that will work also on Win32 machines; it is based on an associative array in order to kill duplicate entries. What's nice with it is that if you add to the printing loop $hDat{$d} you get to know how many times a line was repeated. 
    use strict;
my %hDat;
my $d;

map( $hDat{$_}++, <DATA> );

foreach $d (sort keys %hDat) { print $d; }

__DATA__
a
b
c
dd
c
aa
zzzz
q
r

    [download]
    You could also use a similar approach to perform a case-insensitive duplicate line removal that returns the last instance of the duplicate line in the full majesty of its original case: 
    use strict;
my %hDat;
my $d;

map( ($hDat{lc $_} = $_), <DATA> );

foreach $d (sort keys %hDat) { 
    print $hDat{$d}; 
}

__DATA__
a
Bongo
c
BoNgo
dd
c
A
zzzz
q
r

    [download]
[reply]
[d/l]
[select]
      This is a somehow shorter version using the same technique but no named temporary array (it's more an exercise in concision than actually useful). 
      use strict;

foreach my $d (sort keys %{{ map( ($_ => 1), <DATA>) }}  ) { 
    print $d;
}

__DATA__
a
B
a
B
dd

      [download]
      I wonder if there's a cleaner way to write the mapped expression.
[reply]
[d/l]
[select]
Re: Sorting Unique File Entries
by Abigail-II (Bishop) on Nov 24, 2003 at 11:40 UTC 
    system "sort -u $file > $file.$$; mv $file.$$ $file";

    [download]

    Abigail

[reply]
[d/l]
Re: Sorting Unique File Entries
by moxliukas (Curate) on Nov 24, 2003 at 11:37 UTC

    File::Sort might possibly be what you are looking for.

[reply]

Back to Seekers of Perl Wisdom