Hi Anonymous Monk, I have had quite a thought about it and I think the thing I would most likely do is to first reduce complexity by removing all duplicated lines with the same integer infront of them. Is this right. I tried running the script below and then ran the script above -jgg.pl. But the result are not good. Any ideas or suggestions how I could marry the two in a sensible fashion

my $file = 'my_data_file.txt';

my %seen = ();

{
      local @ARGV = ($file);

      #local $^I = '2.txt';
  
      while(<>){
  
      $seen{$_}++;
  
      next if $seen{$_} > 1;

      print;
  
      }
  
      }
[download]

Hi Anonymous Monk, I have had quite a thought about it and I think the thing I would most likely do is to first reduce complexity by removing all duplicated lines with the same integer infront of them. Is this right.

Based on your descriptions it sounds right.

I tried running the script below and then ran the script above -jgg.pl. But the result are not good.

What is wrong with the results?

Any ideas or suggestions how I could marry the two in a sensible fashion

First get your code to do what you want, then think about marriage

I have edited the script above as follows (below): it writes out a temporary file that I will use while running my second script. The second script will now take the temporary file which I will delete afterwards. Please critisize as it must be 100% accurate.

#!/usr/bin/perl
use strict;
use warnings;

my $file = 'my_data_file.txt';

my %seen = ();

{
      local @ARGV = ($file);

#remove all previous re-organized files
my $remove = "my.txt.tmp";
    if (unlink($remove) == 1) {
                        print "Existing \"$remove\" file was removed\n
+";
                              }
  
      while(<>){
            
  #now make a file for the ouput
       my $outputfile = "my.txt.tmp";
           if (! open(POS, ">>$outputfile") ) {
            print "Cannot open file \"$outputfile\" to write to!!\n\n"
+;
                exit;
        }
   
      $seen{$_}++;
      
  if($seen{$_} !~/-/g){
      
      next if $seen{$_} > 1;

      print POS;
      }
    }
  }
  
  print "\n\nfinished processing file.\n";
[download]