Marshall, that's a good point regarding db-operations. You are right that there is memory cost for the performance boost of using refs with this specific use-case: db-IO. I was talking more generally so probably I gave the wrong impression for this specific case. Anyway refreshing on the manual is good.

According to fetchall_arrayref of DBI a tradeoff would be to fetchall_arrayref() in batches but still use array-refs, as

That might be the fastest way to fetch and process lots of rows using the DBI, but it depends on the relative cost of method calls vs memory allocation.

my $rows = []; # cache for batches of rows
while( my $row = ( shift(@$rows) || # get row from cache, or reload ca
+che:
                   shift(@{$rows=$sth->fetchall_arrayref(undef,10_000)
+||[]}) )
) {
  ...
}
[download]

bw, bliako

bliako, Most excellent reference!

I am an "old school hardware guy", dating back to the dark ages where 64K bytes of iron core memory was a lot! My current machine is 64 bits with 16 GB of semiconductor memory. Memory usage is something to be aware of, but I don't worry about it often, but "seldom does not mean never".

I suppose that the unwary could get into a "memory thrashing" situation with a huge allocation of virtual memory. With my experience so far, if the result set is so humongous that it won't fit into physical memory, then there may be a problem with the SQL query itself! That is something to be considered also!