in reply to Re^2: Sort large files
in thread Sort large files

Sort::Maker memory requirements are quite high so I don't think it could be a good solution.

Sort::External does the sorting on disk, so it is not limited by the memory size and it is very easy to use for simple cases but for complex cases a transformation similar to the GRT is required.

Another way is to use the external sort program that uses on-disk sorting algorithms:

# untested ... use Encode qw(encode decode); use MIME::Base64 qw(encode_base64); my $tempfn = "sort.temp"; # better use File::Temp! open my $tmp, ">", $tempfn or die ...; while(<>) { my $k0 = encode_base64(encode(cp1047 => substr($_, 3, 17)), ""); my $k1 = encode_base64("\xff\xff" ^ encode(cp1047 => substr($_, 20, +2))); print $tmp join("\0", $k0, $k1, $_); } close $tmp or die "..."; open my $sorted, "-|", sort => $tempfn or die "..."; while(<$sorted>) { print((split /\x00/, $_, 3)[2]); }

Replies are listed 'Best First'.
Re^4: Sort large files
by Moron (Curate) on Jun 06, 2007 at 17:57 UTC
    I was going to say, yes, but what about sort keys, until I just-in-time found it explained in Sort::External::Cookbook - so I now agree with you!
    __________________________________________________________________________________

    ^M Free your mind!