It would be better to fetch one row at a time, so long as you are actually processing one row at a time. If you simply fetch one row at a time and build a large array in memory, and then use foreach, you won't be doing yourself any favors. You likely want to use a while loop around your call to fetchrow_arrayref, and operate on each row as it comes out of the database. That way your memory footprint will be minimized.

As to the speed, if your process is forced to page to disk because you're using huge amounts of RAM, it's likely that simply reducing the memory footprint will result in a significant improvement in speed. If you have 1.5GB of data in the DB, you can expect that perl could use three times that much (as a rough figure) to keep that in perl variables/data structures in memory. At the very least, it is worth it for you to try it for yourself and find out.

In reply to Re: fetchall_arrayref DBI question ? by bellaire
in thread fetchall_arrayref DBI question ? by baxy77bax

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.