It looks like you're trying to do too much in your code. If you're goal is to just fill in NULL values, then focus on that first.

The below code removes the uniqueness time check that you were doing since it appears superfluous, and relies on the fact that the times are sequential instead of having to do a sort. In the end everything is the same as your code though, the data structures are just simplified and all the validation is done in a single loop

use strict; use constant NULL_VAL => -999.9; my @data = (); while (<DATA>) { chomp; s/^\s+|\s+//g; next if $_ eq ''; push @data, [split ',']; } my %last_good = map {$_ => NULL_VAL} (1..$#{$data[0]}); # Go in reverse to fill in null data for (my $i=$#data; $i>=0; $i--) { # Validate each column for my $j (1..$#{$data[$i]}) { if ($data[$i][$j] == NULL_VAL) { $data[$i][$j] = $last_good{$j}; } else { $last_good{$j} = $data[$i][$j]; } } } foreach (@data) { print join(',', @$_) . "\n"; } # time,col1,col2 # (-999.9 out-of-range to indicate missing # and distinguish from true zeroes) __DATA__ 0.01,-999.9,1 0.02,-999.9,-999.9 0.03,-999.9,3 0.04,2,-999.9 0.05,-999.9,-999.9 0.06,5,4
Note, this is still potentially fragile in a way though, since it requires loading everything in memory to do the update. But you don't say whether you're doing anything special with this data, like outputting it to a new file.

In reply to Re: How best to fill missing values in a sparse matrix? by wind
in thread How best to fill missing values in a sparse matrix? by astrobio

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.