halligalli has asked for the wisdom of the Perl Monks concerning the following question:

I am doing a quick data evaluation. First I made a simple check if the measured values are within a possible range (see code).After that I would like to detect "frozen values". I have a value, and if the next 5 following measurements have exactly the same value I would like to flag them. Does anybody has an idea how to code it?
# Station Loop for ($i=0;$i<=$#data;$i++) { # check if value is available else set flag 9 if ($channel[$i][$value_index] eq '') {$QV[$i]=9} # Range Check if ($N <= 300) {$MIN_TOL=2.5; $MAX_TOL=45.5}; # Flagging if ($channel[$i][$value_index] < $MIN_TOL || $channel[$i][$value_index +] > $MAX_TOL ) { $QV[$i]=4 } else {$QV[$i]=2} } ###"Stuck Value Test"??? } }
Thanks in advance

Replies are listed 'Best First'.
Re: Stuck Value
by Corion (Patriarch) on Dec 02, 2010 at 12:57 UTC

    How would you do it if you had no computer and had the measured values printed on a sheet of paper?

    How would you do it if you had no computer, had the measured values printed on a sheet of paper, but could only ever read one line of paper at a time?

    You are allowed to write on additional sheets of paper.

Re: Stuck Value
by jethro (Monsignor) on Dec 02, 2010 at 13:22 UTC
    Just an observation:

    Your code seems to make a range check, whether the value is between 2.5 and 45.5, but only if $N is smaller than 300. If $N is greater than 300, $MIN_TOL and $MAX_TOL seem to be undefined (if you haven't set default values somehwere else in the surrounding code). If not, the value is flagged if it is greater or lower than UNDEF aka 0, which is true for all values except 0.

    If you turn on warnings (if you haven't already, do so), perl would warn you about such problems.

Re: Stuck Value
by JavaFan (Canon) on Dec 02, 2010 at 13:24 UTC
    Assuming @data is the array you want to check for 5 indentical (string) values in a row:
    my $c = 0; for (my $i = 1; $i < @data; $i ++) { if ($data[$i] eq $data[$i - 1]) { if (++$c >= 4) { print "Five indentical values in a row, starting at positi +on ", $i - 4, "\n"; } } else { $c = 0; } }
Re: Stuck Value
by GrandFather (Saint) on Dec 02, 2010 at 20:23 UTC
    # Stuck value check for my $index (0 .. $#data - 5) { my $first = $data[$index]; next if grep {$first != $_} @data[$index + 1.. $index + 5]; # Here if a 'stuck' value was found }

    Note, you should always use strictures (use strict; use warnings;). Use the Perl for loop rather than the C variant - it's a lot clearer and a lot safer.

    You iterate over a range in terms of @data, but you never access the array. If @data, @channel and @QV are related data use one data structure to represent them, otherwise there are too many opportunities for the data in the various arrays to get out of sync.

    Your range check looks like it should be outside the Station loop.

    True laziness is hard work
Re: Stuck Value
by anonymized user 468275 (Curate) on Dec 02, 2010 at 13:11 UTC
    ...and beware that you can't test the first set of values for non-changing. Further, people already think you want to test local pairs of instances for stuck values -- it is more likely you want to check for similarity over more than two instances before flagging it stuck. How many instances of no-change is enough to be declared stuck?

    One world, one people

      I am processing environmental variables (hourly measurements) and we declared variables as stuck if we have 5 instances of no-change.
        ok then to index the last five elements of an array say @arr up to and including $i, use the array slice
        $arr[ ($i - 4) .. $i ]
        update: and to avoid trying to index below 0: and suggesting an inequality function:
        my $h = $i - 4; if ( $h >= 0 ) { if (unequal ( $arr[ $h .. $i ] ) { # flag stuck value } } # ... sub unequal { my %v = map { ($_, 1 ); } @_; my @v = keys %v; return $#v; }

        One world, one people

Re: Stuck Value
by locked_user sundialsvc4 (Abbot) on Dec 02, 2010 at 13:37 UTC

    I would do it in the simplest way possible:   the first data-structure that comes to mind.   This would be a hashref (of keys...) pointing to an arrayref (of values).

    Each time you get a value, you unshift it onto the array.   If the number of entries in the array is now 6, shift off the first one and discard it, then check for identical runs.   (If not, you don’t have enough observations yet.)

    The test for runs is also very straightforward:   a while loop.

    In short, there is no reason for your coding to be “clever.”   (You are in no danger of running out of microseconds.)   The only thing that matters is that it is clear and maintainable.

Re: Stuck Value
by halligalli (Novice) on Dec 03, 2010 at 07:50 UTC
    thanks for your help