As my duplicate lines tend to be clustered I have tried this code:
while(<INFILE>){ #print "here\n"; if ($_ =~ /^(.{1,50})\t{0,50}(.{0,50})\t{0,1}(.{0,50})\t{0,1}(.{0, +50})\t{0,1}(.{0,50})\t{0,1}(.{0,50})\t{0,1}(.{0,50})\t{0,1}(.{0,50})\ +t{0,1}(.{0,50})/){ #print "here_B\n"; $bumb = ""; $gene_id = $1; if ($2 =~ /.{1,50}/){ push (@alternative_ids, $2)} else{ push (@alternative_ids, $bumb); } if ($3 =~ /.{1,50}/){ push (@alternative_ids, $3)} else{ push (@alternative_ids, $bumb); } if ($4 =~ /.{1,50}/){ push (@alternative_ids, $4)} else{ push (@alternative_ids, $bumb); } if ($5 =~ /.{1,50}/){ push (@alternative_ids, $5)} else{ push (@alternative_ids, $bumb); } if ($6 =~ /.{1,50}/){ push (@alternative_ids, $6)} else{ push (@alternative_ids, $bumb); } if ($7 =~ /.{1,50}/){ push (@alternative_ids, $7)} else{ push (@alternative_ids, $bumb); } if ($8 =~ /.{1,50}/){ push (@alternative_ids, $8)} else{ push (@alternative_ids, $bumb); } if ($9 =~ /.{1,50}/){ push (@alternative_ids, $9)} else{ push (@alternative_ids, $bumb); } print @alternative_id; foreach (@alternative_ids){ $old_line = $new_line; $alternative_id = $_; $new_line = "$gene_id\t$alternative_id\n"; @record_previous_lines = qw/blank/; if ($new_line ne $old_line){ #discount consecutive repeats of +identical lines $switch = 1; foreach(@record_previous_lines){ if ($_ =~ /$new_line.$old_line/){ $switch = 2; } } if ($switch = 1){ # don't print lines that have previously bee +n repeated print OUT_B "$new_line$old_line"; } if (@record_previous_lines > 200){ # don't let this array get +bigger than 200 shift @record_previous_lines; # shift deletes the first va +lue of an array } push (@record_previous_lines, $new_line.$old_lines); } } undef @alternative_ids; } }
The last bit doesn't seem to be working as I want. Can anyone thow light on this? Sorry I am not being more specific.

In reply to Re: Re: Removing repeated lines from file by matth
in thread Removing repeated lines from file by matth

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.