in reply to Select DB data into CSV file

Using an AoA from Text::CSV_XS is very inefficient. Please read the examples that show you how to do this streaming.

I should update that section to make use of the relative recent undef_str attribute.


Enjoy, Have FUN! H.Merijn

Replies are listed 'Best First'.
Re^2: Select DB data into CSV file
by 1nickt (Canon) on Dec 20, 2018 at 13:06 UTC

    Sorry, sloppy writing. I meant this: 

    # using the csv function, all in memory
csv (out => "foo.csv", in => $dbh->selectall_arrayref ($sql));

    [download]


    The way forward always starts with a minimal test.
[reply]
[d/l]

      This is fine until your table outgrows your memory: selectall_arrayref will read all rows into memory and then pass it to csv (), whereas using a fetch handle has virtually no memory overhead.

      my $sth = $dbh->prepare ("select * from foo");
$sth->execute;
csv (out => "foo.csv", in => sub { $sth->fetch }, undef_str => "\\N");

      [download]

      Enjoy, Have FUN! H.Merijn
[reply]
[d/l]
[select]

        Yes, understood. That's standard DBI practises ... I'm dealing with a small table, so no memory concerns. Mostly wondering if there was a sugary interface, and more especially, in the context of DBIx::Class.


        The way forward always starts with a minimal test.
[reply]
[d/l]

In Section Seekers of Perl Wisdom