Update: Alternatively, I'd use a database, maybe either DB_File or DBI/DBD::SQLite. But sort -u is probably the simplest solution.

Another update: I now like MSYS for my Unix utilities

perl -e "my %hash; @hash{<>} = 1; print sort keys %hash;"
[download]

The op mentioned this was a huge file. Reading the entire thing into a hash, then performing 2 copies on the data (sort keys) is probably going to be very painful depending on how huge is huge.

Please see Re: UNIX shell commands in Windows for what I hope is a fairly comprehensive list of *nix-like utilities for DOS/Windows.

HTH,

We have sort and uniq so it's a matter of combining them:

use List::MoreUtils qw/uniq/;
my @a = qw/4 6 8 9 3 1 5 8 9 0 0 2 5 7 9 4 2 5 5 4 7 9 6 4 2/;
my @s = uniq sort @a;
print "s: @s\n";
__END__
s: 0 1 2 3 4 5 6 7 8 9
[download]

perl -ne "next if $seen{$_}++;print" a.txt | sort >b.txt
[download]

I wonder is there any way I can write Perl program, which would use any inbuilt function to create 'b.txt' on Windows itself.

In the spirit of TIMTOWTDI!:

#!/usr/bin/perl -w 
use strict;
ues Tie::File;

my @ry=();
tie @ry,"Tie::File","a.txt" or die $!;
my %een=();
@ry = grep !$een{$_}++,sort @ry;
untie @ry;
[download]

HTH

Install Sort::External from CPAN. It was written for the purpose of sorting huge files.

    my $sortex = Sort::External->new();
    while (<HUGEFILE>) {
        $sortex->feed($_);
    }
    $sortex->finish;
    my $prev = '';
    while ( defined( $_ = $sortex->fetch ) ) {
        next if $_ eq $prev;
        print OUTFILE $_;
        $prev = $_;
    }
[download]