I would guess that DBI would process all 50000 records reguardless of what you do with the data. I am not posivite about that though. The safest bet for database performance is to always try to increase your query performance. (i.e. if you only need 20 rows, then only retrieve 20 rows). That way DBI will not have to do all the extraneous processing.

In sybase there is a command "set rowcount NUMBER" which will only retrieve the first NUMBER rows. There may be something similar for the database you are using.

So if you know you are only going to use the first 20 rows of a 50000 row dataset, then you might be able to limit it in your query:

my $sql =<<EOF;
set rowcount 20
SELECT * from total_huge_table
EOF
my $sth = $dbh->prepare($sql);
$sth->execute();
... more code here ...
[download]

If this kind of query restriction is available via a perl module, I would guess that it would be mentioned in the individual DBD module perldoc pages. I dont think this could be a DBI feature since it is probably really database dependent.

I just noticed that Mysql has a way to limit in the SQL also: SELECT * from huge_table LIMIT 1, 20This will return the 1st through 20th rows. (untested, but that is what the O'Reilly mysql book says: MySQL & mSQL by Yarger, Reese, & King)

Well I dont think I answered you exactly, but maybe some of this will help.