in reply to Remove row if the absolute difference between two columns is greater than a threshold

grep -v is an acceptable Unix command line method to exclude a line, but it does not qualify as a Perl solution ... at least not as a GOOD perl solution. Besides which, that would drop only one line from the file. Imagine a worst case in which you wind up excluding every line in a 1000 line file ... You would have to copy the file, sans one line, 1000 times.

What you want is to go through the file, line by line, using open(), while(), and close(), test each line, and if acceptable, copy it to the output file. That means only one copying of the file, whether you drop 0 lines or a million.

You say "the absolute value of column 2 minus column 2 ( I guess you mean column 3 ) is greater than or equal to 1". Except for the absolute value bit, I would test for $col2 > $col3. But it's significantly different whether you mean abs( $col2 ) > abs( $col3 ) or whether you mean abs( $col2 - $col3 ).

As Occam said: Entia non sunt multiplicanda praeter necessitatem.

  • Comment on Re: Remove row if the absolute difference between two columns is greater than a threshold
  • Download Code

Replies are listed 'Best First'.
Re^2: Remove row if the absolute difference between two columns is greater than a threshold
by Renyulb28 (Novice) on Feb 15, 2011 at 17:22 UTC
    thank you for the reply. I do mean abs(column 2 - column 3). I would like the script to be able to either remove those rows in which that absolute value is greater than or equal to 1.