Re^2: Large file, multi dimensional hash

Replies are listed 'Best First'.
Re^3: Large file, multi dimensional hash - out of memory by BrowserUk (Patriarch) on May 15, 2013 at 15:10 UTC
As the input file is sorted, `uniq infile > outfile` ought to do the job very quickly. If for some reason you don't have the uniq command available, try `#! perl -w use strict; my $last = <>; while( <> ) { print $last if $_ ne $last; $last = $_; } __END__ thisscript infile > outfile` [download] With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: Large file, multi dimensional hash - out of memory
by BrowserUk (Patriarch) on May 15, 2013 at 15:10 UTC

As the input file is sorted, uniq infile > outfile ought to do the job very quickly.

If for some reason you don't have the uniq command available, try

#! perl -w
use strict;

my $last = <>;
while( <> ) {
    print $last if $_ ne $last;
    $last = $_;
}

__END__
thisscript infile  > outfile
[download]

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

[reply]
[d/l]
[select]