ok, I'm at my work place so cannot submit the code right now but I'll post my algorithm at least because my boss is not around and this post got me ticking... ;).
I'll upload my rendition of the code once I get home.

A Sparse Matrix

A matrix populated mostly with 0s (or null for objects) is called a sparse matrix. Refer this. Keeping this in mind and re-reading your question, I can represent your data as a sparse matrix (critical analysis of this assumption is welcome).

To parse a sparse matrix, my algorithm is as follows:

# ALGORITHM: parseSparse (M) // M is the given matrix

    //first, we "incrementally" parse the matrix
    //this enables us to capture only non-zero entries
    //of each row.

    set sum = 0;

    foreach $row{
        foreach $col{
            if M($row, $col) != 0
                $key = concatenate ($row, $col);
                $hash{$key} = M($row, $col);
        }
    }

    //now that my hash is ready, I can do random access faster

    /* implement what you want to implement, I will just sum all the n
+on zero elements of M */

    foreach $key (keys %hash) {
      sum += $hash{$key};
    }
[download]

The algorithm essentially creates a 3-tuple (row, column, value) and using a hash makes random access faster as compared to arrays. The wiki link above elucidates more algorithms to represent a sparse matrix.