In Perl, the rule is: When in doubt, use a hash!

So, create an empty hash. Then read the data file, one record at a time: if its date/time is not already a key in the hash, store whatever values you need against this new key. If the date/time is already in the hash, compare the QC flags — if the new record has a smaller value, substitute the new record’s details for those currently stored against that date/time.

When all records have been read, output the hash contents, sorted by their date/time keys.

Hope that helps,

Is your data sorted for date/time, i.e. are all records with the same date/time together? You implied it but I want to make sure I understand you correctly.

In that case you can just write a loop, reading each record and checking for two things: If the date/time is the same as the previous record then if the QC flags are lower than the QC flags of a temporary record, replace the temporary record with this record. If the date/time is new, then print the temporary record (which holds the lowest record of the previous date/time) and replace the temporary record with this record. That's all

If your data is not sorted for date/time, then you need to store the smallest record for each date/time in a hash and output those records after you read the whole file. A problem might be that you need to have as many records as you have different date/times in memory, but it doesn't sound as if that is a problem for your case

PS: A flag is a data item that is either on or off. So technically there is no smallest value for flags. You can easily confuse programmers with misnomers like this

i think, it is better to take a look at your file, first. however, i can have a recommendation. take date/time as a hash key and use references for a complex array in array.

use strict; use warnings;
use List::Util qw(min);

my %seahash; my $date; my $time; my $coolflag; 

while(<DATA>){
 next if /^date\/time/;
 chomp;
 my ($date, $time, $lat, $lon, $depth, $flag, $coolflag, $otherflag) =
+ split;
 # you could need the other variables later, so I save it
 # print $date, " ",$time, "  ", $coolflag,"\n"; # checkpoint
 push @{$seahash{$date . " " . $time}}, $coolflag;
}

while (my ($timestamp, $flags) = each (%seahash)) { 
            print $timestamp, " ", min(@{$flags}),"\n"}

__DATA__
date/time             lat, lon, depth, flag, flag, flag
21-sep-2012 01:00:00  0    0    -200    a    1     c
21-sep-2012 01:00:00  0    0    -100    a    9     c
20-sep-2012 02:20:00  0    0   -1500    a    2     c
20-sep-2012 02:20:00  0    0    -500    a    4     c
20-sep-2012 02:20:00  0    0    -400    a    3     c
20-sep-2012 02:20:00  0    0    -300    a    3     c
10-Aug-2012 02:20:00  0    0    -200    a    3     c
10-Aug-2012 02:20:00  0    0    -200    a    1     c
10-Aug-2012 02:20:00  0    0    -200    a    7     c
[download]