Do those statements only once, and maintain your data in arrays. Disk operations should be minimal, and in your case you only need to read and write once.
--
bm

my %seen;
while (<DATA>) {
    /^([^\|]+)\|(.*)/; # split at the first "|"
    my ($key, $value) = ($2,$1);
    next if $seen{$key};
    print "$key: $value\n";
    $seen{$key}++;
}
__DATA__
34|4|45|56|45
45|34|45|00|23
45|34|45|00|27
34|4|456|56|03
36|4|456|56|03
[download]

Good use of a "seen" hash, asarih, but i have to say that i would rather use this:

my ($value,$key) = $_ =~ /^([^\|]+)\|(.*)/;
[download]

than explicitly "spell out" $1 and $2. But if all you want to do is split at the first pipe, just use split:

my ($key,$value) = split(/\|/,$_,2);
next if $seen{$value};
print "$key: $value\n";
$seen{$value}++;
[download]

I don't have time for some benchmarking right now, but substr and index are fast:

my $index = index($_,'|');
my $key   = substr($_,0,$index);
my $value = substr($_,$index);
[download]

Just some more ways to do it. ;)

jeffa

L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)

I tried as suggested but cant get your new script to work on my text file. Please advise what Iam doing wrong. Thanks.

my $db = 'C:\Inetpub\wwwroot\cgi-bin\test4.txt';
open(DATA, "$db") || die "Can not open: $!\n";
my @dat = (<DATA>);
close(DATA);
open(DATA, "$db") || die "NO GO: $!\n";
my %seen;
while (<DATA>){
my ($value,$key) = $_ =~ /^([^\|]+)\|(.*)/;
my ($key,$value) = split(/\|/,$_,2);
next if $seen{$value};
print "$key: $value\n";
push(@files,$key);
my @files = (<DATA>);
print DATA @files;
$seen{$value}++;

}


close(DATA);
[download]

Still cant get it to write to file:

my $db = 'C:\Inetpub\wwwroot\cgi-bin\test4.txt';
open(DATA, "$db") || die "Can not open: $!\n";
my @dat = (<DATA>);
close(DATA);
open(DATA, "$db") || die "NO GO: $!\n";


my %seen;
while (<DATA>) {
    /^([^\|]+)\|(.*)/; # split at the first "|"
    my ($key, $value) = ($2,$1);
    next if $seen{$key};
    print "$key: $value\n";
     $seen{$key}++;
    push(@files,$key);
    pop @files;
         open(DATA,"> test4.txt") or die $!;
        print DATA @files;
}




close(DATA);
[download]

There are all sorts of problems with this code. You seem to have copied snippets from various answers to your question into your code without understanding any of them. I suggest you spend some considerable time studying the resources listed here.

However, to return to your immediate problems, let's look, for example, at the number of times you use open in your code. Lines 3-7 are as follows:

open(DATA, "$db") || die "Can not open: $!\n";
my @dat = (<DATA>);
close(DATA);
open(DATA, "$db") || die "NO GO: $!\n";
[download]

Why on earth do you want reopen (for reading) the same file that you've just closed, once you've already read it into an array?

As perl is forgiving, you can needlessly open (and/or close) the same file as many times as you want without it complaining, but...

More importantly, the third time you use open is inside a while loop:

while (<DATA>) {
   # ...
   open(DATA,"> test4.txt") or die $!;;
   # ...
}
[download]

Apart from the fact that - at least for clarity's sake - you shouldn't be using the same filehandle for two completely different files (and that in any case DATA is not a particularly good filehandle to choose...) - try to envision what the above is doing. As the open line is inside a loop, it will, for each iteration of the loop, open 'test4.txt' for overwriting. If you don't understand that, try running this:

my $i = 0;
while ($i < 5) {
   open (OUT, '>oops.txt') || die "NO GO: $!\n";
   print OUT $i;
   $i++;
}
[download]

as compared with this:

open (OUT, '>oops.txt') || die "NO GO: $!\n";
my $i = 0;
while ($i < 5) {
   print OUT $i;
   $i++;
}
[download]

dave

Then again, you could simply use Tie::File and grep, but where's the fun in that? :)

Cheers, Ben.

Here's the fun: SPOILER ;)

jeffa

L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)

#!/usr/bin/perl

use strict;
use warnings;

my ($cur_rec, $prev_rec, $line) = ("", "", "");

while ($line = <STDIN>) {
    $cur_rec = join("", (split(/\|/, $line))[1..4]);
    if ($cur_rec ne $prev_rec) {
        print $line;
    }
    $prev_rec = $cur_rec;
}
[download]

Arjen