Something like this?
#!/usr/bin/perl
use strict;
use warnings;
my ($cur_rec, $prev_rec, $line) = ("", "", "");
while ($line = <STDIN>) {
$cur_rec = join("", (split(/\|/, $line))[1..4]);
if ($cur_rec ne $prev_rec) {
print $line;
}
$prev_rec = $cur_rec;
}
With this script, you can process files of arbitrary length, because the while file isn't stuffed into a hash. It processes from standard input to standard output, thus you can give it any file you like, instead of hardcoding filenames in the script. It's a lot more efficient than opening, writing, and closing the same file over and over in a loop.
Arjen