I would guess that DBI would process all 50000 records reguardless of what you do with the data. I am not posivite about that though. The safest bet for database performance is to always try to increase your query performance. (i.e. if you only need 20 rows, then only retrieve 20 rows). That way DBI will not have to do all the extraneous processing.

In sybase there is a command "set rowcount NUMBER" which will only retrieve the first NUMBER rows. There may be something similar for the database you are using.

So if you know you are only going to use the first 20 rows of a 50000 row dataset, then you might be able to limit it in your query: 
my $sql =<<EOF;
set rowcount 20
SELECT * from total_huge_table
EOF
my $sth = $dbh->prepare($sql);
$sth->execute();
... more code here ...

[download]
If this kind of query restriction is available via a perl module, I would guess that it would be mentioned in the individual DBD module perldoc pages. I dont think this could be a DBI feature since it is probably really database dependent.

I just noticed that Mysql has a way to limit in the SQL also: SELECT * from huge_table LIMIT 1, 20This will return the 1st through 20th rows. (untested, but that is what the O'Reilly mysql book says: MySQL & mSQL by Yarger, Reese, & King)

Well I dont think I answered you exactly, but maybe some of this will help.

In reply to Re: More on selecting rows with DBI... by perlmonkey
in thread More on selecting rows with DBI... by BBQ

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.