As long as this thread has gotten so popular, I'll throw in my 2 cents + TMTOWTDI. No matter what your final intent is, the problem you've presented is just shredding a space-delimited file (as pointed out by JavaFan and jethro). I present my solution:

use strict;
use warnings;

my (@A, @IA, @JA) = ();
while (<DATA>) {
    chomp;
    my @elements = split /\s+/;
    my $i = 0;
    my $new_line = 1;
    while (defined(my $element = shift @elements)) {
        $i++;
        if ($element) {
            push @A, 0 + $element;
            if ($new_line) {
                push @IA, scalar @A;
                $new_line = 0;
            }
            push @JA, $i;
        }
    }
}
push @IA, 1 + @A;
print('@A  = [', join(" ", @A), "]\n");
print('@IA = [', join(" ", @IA), "]\n");
print('@JA = [', join(" ", @JA), "]\n");


__DATA__
1 2 0 0
0 3 9 0
0 1 4 0
[download]

As a side note, if you are planning on doing actual math on the results, you should plan on using seriously optimized math libraries (read LAPACK, ATLAS, Intel mkl...)