my (%count, %unique);
while (<IN>) {
  print DUP if $count{$_}++;
  if ($count{$_} == 1) { $unique{$_} = 1 }
  else { delete $unique{$_} }
}
print UNQ for sort keys %unique;
[download]

_____________________________________________________
Jeff[japhy]Pinyan: Perl, regex, and perl hacker, who'd like a job (NYC-area)
s++=END;++y(;-P)}y js++=;shajsj<++y(p-q)}?print:??;

Since the input is sorted, you just need one record of "lookahead":

use strict;
my $prev;
my $dup=0;
my ($ifile, $ufile, $dupfile) = qw(data uniq dups);

open(IN, $ifile) or die "Cannot open $ifile: $!\n";
open(UNQ,">", $ufile) or die "Cannot open $ufile: $!\n";
open(DUP, ">", $dupfile) or die "Cannot open $dupfile: $!\n";

while (<IN>) {
  if (defined($prev)) {
    if ($prev eq $_) {
      $dup = 1;
      print DUP $prev;
    } else {
      if ($dup) {
    print DUP $prev;
    $dup = 0;
      } else {
    print UNQ $prev;
      }
    }
  }
  $prev = $_;
}
  if (defined($prev)) {
      if ($dup) {
    print DUP $prev;
    $dup = 0;
      } else {
    print UNQ $prev;
      }
  }
[download]

Theo, this solution, if I'm parsing it correctly, assumes that all duplicate records are 'stacked' together... ie, all 1111 lines occur one after another, correct? I pretty much discarded that pattern as soon as I saw it, since rarely do the problem sets order themselves up that conveniently. ;)

If that pattern is guaranteed, then yes, the hash is a waste of memory. If it's not, then your solution... isn't. Personally, I'd rather eat the memory usage and feel comfortable knowing I didn't have to rely on my input to match my expectations, which are fraught with danger and ignorance most days.

If the memory usage of the hash is that problematic (and let's be honest... when folks say "I only had X amount of memory to use!" it's not because that was all they *wanted* to use, now was it? :) , then stash results off in some other manner (DBI leaps to mind) and read in a limited set of lines at a time.

Theo, this solution, if I'm parsing it correctly, assumes that all duplicate records are 'stacked' together... ie, all 1111 lines occur one after another, correct? I pretty much discarded that pattern as soon as I saw it, since rarely do the problem sets order themselves up that conveniently. ;)

He said in the specification that the input file was already sorted.

...
while (<IN>) {
    $seen{$_}++;
}

foreach my $id (sort keys %seen) {
    if ( $seen{$id} == 1 ) {
        print UNQ "$id\n";
    }
    else {
        print DUP "$id\n";
    }
}
...
[download]

A dup needs printed to the file each time it appears.

(Untested)

...
while (<IN>) {
    $seen{$_}++;
}

foreach my $id (sort keys %seen) {
    if ( $seen{$id} == 1 ) {
        print UNQ "$id\n";
    }
    else {
        print DUP "$id\n" foreach 1..$seen{$id};
    }
}
...
[download]

The sort keys %seen offered here would render the outputs sorted, not in the order they were originaly found. This may or may not be suitable.

You can either keep a separate array of all unique items in the order discovered, or you can look at Tie::IxHash for an already-bundled solution.

--
[ e d @ h a l l e y . c c ]

Note: I know this is Seekers of Perl Wisdom, but there were already plenty of good Perl solutions, and I couldn't resist.

If you're on a unix (should I say GNU?) machine, or have cygwin installed on a Windows machine, you can very easily accomplish these tasks with sort and uniq.

Print unique lines:

Print all duplicates:

(Or if you know the numberlist is already sorted, uniq -D numberlist will do.)

Almost right. sort -u will give one copy of each input record whether it was unique or not to begin with. What he wants is uniq -u and uniq -D since he stated the input was already sorted. My program below does it in one pass, though.

Well, sort -u (or simply the default behavior of uniq) matches the behavior of the OP's sample code -- that is, it prints all values once, but only once. Perhaps that is not what he wanted, but if he stated that in his original post, it's not clear to me. Either way, it seems he has many solutions to any number of problems, some of which may be the actual problem he was looking for help with. :)

my %hash;
while ( <IN> ) {
    $hash{ $_ }++;
}

foreach my $key ( sort keys %hash ) {
    if ( $hash{ $key } > 1 ) {
        print DUP "$key\n" x $hash{ $key };
    } else {
        print UNQ "$key\n";
    }
}
[download]

I suggest that you create a hash to store the number of times you have seen something. Then, you can iterate over the hash to create the output.

Updated: So, I was a little behind on this and basically solved it the same way as other monks. Oops. That's what happens when your boss walks in while you are playing around on PM. :)

use strict;

open (IN, "test.txt")   or die "Can't open file: $!";
open (UNQ, "> unq.txt") or die "Can't open file: $!";
open (DUP, "> dup.txt") or die "Can't open file: $!";

my %hash;

while (<IN>) { chomp; $hash{$_}++; }

foreach (keys %hash) {
  if ( $hash{$_} > 1) { print DUP "$_\n"; }
  else { print UNQ "$_\n"; }
}

__END__

fraser:~$cat test.txt
0000
1111
2222
3333
4444
0000
3333
1111
5555
1111
0000
0000
1111
3333
6666
0000
fraser:~$cat unq.txt
6666
4444
2222
5555
fraser:~$cat dup.txt
0000
3333
1111
fraser:~$
[download]

</ajdelore>

In this code the line:

    ($cur, $next) = ( $next, scalar( <IN>));
[download]

    @queue[0 .. 4 ] = ( @queue[ 1 .. 3], scalar( <IN> ));
[download]

This code seems to work.

#!/usr/bin/perl -T
use strict;
use warnings;

my ( $input, $unique, $dupe) = qw( input unique dupes );
my $is_dupe;

open( IN, $input )           or die "Can not open $input";
open( UNIQUE, ">", $unique ) or die "Can not open $unique";
open( DUPES, ">", $dupe )    or die "Can not open $dupe";

my $cur = <IN>; 
exit if not defined $cur;

my $next = <IN>;

no warnings "uninitialized";
while ( 1) {
    if ( $cur == $next) {
        print DUPES $cur;
        $is_dupe = 1;
    } else {
        if ( $is_dupe) {
            print DUPES $cur;
        } else {
            print UNIQUE $cur;
        }
        $is_dupe = 0;
    }
    ($cur, $next) = ( $next, scalar( <IN>));
    last if not defined $cur;
}
[download]

Of course, there's no need to write a whole program, compute expensive hashes and us gobs of memory for things that you can do with a nice, easy to remember one-liner :)

perl -ne"print{$*ne$_&&$*ne$@?STDOUT:STDERR}$*if$*;($@,$*)=($*,$_);END
+{print{$*ne$_&&$*ne$@?STDOUT:STDERR}$*}" in 1>uniq 2>dups
[download]

Caveat: Usually OS quoting rule changes.