Is there any implementation for handling such a large matrix to give us A, IA and JA, without having have to slurp all the files into RAM?
As you describe A, IA and JA, it seems to me that each of them can be created by just scanning the matrix top to bottom; left to right. Since that means never having to slurp in more than the length of an element, I don't see why you would have to slurp in the file into memory.

Now, you still may have so many non-zero elements such that you don't have memory enough for A, IA and JA, but that's a different matter.

BTW, are you really counting starting from 1?

In reply to Re: Memory Efficient Sparse Matrix Handling by JavaFan
in thread Memory Efficient Sparse Matrix Handling by neversaint

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.