After sleeping on this, I realized the obvious, which is that each $matrix[$j]->[$i] represents potential file I/O, since %matrix is tied to a DBM. Duh.

I can't believe that building the data structures is taking that large a percentage of the total time, even for that many cells. This simple test

  my @array;

  my $t = time();
  for my $i ( 0 .. 499 ) {
    for my $j ( 0 .. 6264 ) {
        $array[$i]->[$j] = 47;
    }
  }
  print "Elapsed: ", time() - $t, " seconds\n";
[download]

ran in 17 seconds on my 400Mhz laptop, so it has to be the disk I/O that's killing you.

If you're unable to use a different DBM representation (such as DB_TREE), then you might try pulling keys and values out of the DBM in whatever order it prefers to give them to you, and then sort them by key in memory. By pulling out the keys and sorting before going after the values, you might be trashing around a bit in the file.