By declaring your variables inside the loop, they are being redeclared each time the loop executes.

If you move your declarations before the loop, it should work:

#!/usr/bin/perl
use warnings;
use strict;

my $hitlength;
my $prevalence;
my $prev_l;
my $prev_p ;

while(<>){
[download]

Thank you very much Ron, you saved me a lot of hair pulling!

As well as the re-declared variables, it looks like you have a typo on the line:

This should probably be:

You are indeed correct, thank you very much!

#!/usr/bin/perl
use warnings;
use strict;
my $prev_l;
my $prev_p;
my $count = 0; ### add
while(<>){
  my $hitlength;
  my $prevalence;
 
  my $line = $_;  
  chomp $line;
  if ($line =~ /^[\d]/){
    my @hitline = split(/,/ , $line);
    $hitlength  = $hitline[4];
    $prevalence = $hitline[65];
    if( ($count++) and
        ($hitlength  == $prev_l) and
        ($prevalence == $prev_p) ){
        next;
    } else {
      print "$line\n";    
    }
    $prev_l = $hitlength;
    $prev_p = $prevalence;
  } 
}
[download]

You also need to skip the comparison for the first line of data.

Or you could just give an initial dummy value to the two previous variables, thereby saving some steps within the loop:

#!/usr/bin/perl
use warnings;
use strict;
my ($prev_l, $prev_p) = ("", "");
while(<>){
  my $hitlength;
  my $prevalence;
 
  my $line = $_;  
  chomp $line;
  if ($line =~ /^[\d]/){
    my @hitline = split(/,/ , $line);
    $hitlength  = $hitline[4];
    $prevalence = $hitline[65];
    if( $hitlength  == $prev_l and
        $prevalence == $prev_p ){
        next;
    } else {
      print "$line\n";    
    }
    $prev_l = $hitlength;
    $prev_p = $prevalence;
  } 
}
[download]

That'll yield a warning:

c:\@Work\Perl\monks>perl -wMstrict -le
"my $x = '';
 print 'equal' if $x == 1;
"
Argument "" isn't numeric in numeric eq (==) at -e line 1.
[download]

---

Give a man a fish:  <%-(-(-(-<

Apart from the warning, you have to be certain that on the first line of data those 2 fields don't contain the dummy values you initialize to otherwise it will be treated as a duplicate and skipped.


poj

As a general thing don't declare variables until you have an initial value for them. The use of prev_l and prev_p is an exception to that guideline where the operation of the code requires the two variables to be declared outside the loop.

Nested blocks generally obfuscate code so try to avoid that. One powerful tool for reducing nesting is to use early exits. Look at the following code and see how next with if as a statement modifier is used to abort processing loop statements wherever there is no need to continue.

Note too the use of defined to handle the first line correctly.

#!/usr/bin/perl
use warnings;
use strict;

my $prev_l;
my $prev_p;

while (my $line = <DATA>) {
    next if $line !~ /^[\d]/;
    chomp $line;

    my @hitline    = split (/,/, $line);
    my $hitLength  = $hitline[4];
    my $prevalence = $hitline[65];

    next if defined $prev_l && $hitLength == $prev_l && $prevalence ==
+ $prev_p;

    print "$line\n";
    $prev_l = $hitLength;
    $prev_p = $prevalence;
}

__DATA__
23750,57495,78362,xxxx,2853,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,60,100
23751,57497,78364,xxxx,2853,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,60,100
23752,57500,78367,xxxx,3114,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,60,100
23753,57502,78369,xxxx,3114,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,60,100
23754,57504,78371,xxxx,101,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,60,100
[download]

Prints:

23750,57495,78362,xxxx,2853,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,60,100
23752,57500,78367,xxxx,3114,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,60,100
23754,57504,78371,xxxx,101,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+1,1,1,1,60,100
[download]

It is available in Perl 5.10 or later.

Just to clarify ... there are, basically, two errors at work here:

The first error is that variables are declared inside the loop, that need to be declared outside.   Local-variables within a scope (such as “the scope of a while loop”) will be re-initialized each time.   Thus, the value from previous iterations are not retained.

The second error is a little bit more subtle, since it is a logic error:   during the first iteration of the loop, (by definition) there is no “previous record” to be compared against.   Variables such as $prev_1 and $prev_p, therefore, will be undefined.   There are many ways to check for this and to handle it ... as long as you do.   The defined() function, for example, will check for undef.   Therefore, yet-another way to handle this case (in the present program), is to enclose the current if-statement within another if that tests whether (say...) $prev_1 is defined.