zyzzogeton has asked for the wisdom of the Perl Monks concerning the following question:

I noticed in the CPAN docs for DBD:CSV that LongReadLen is not supported with DBD::CSV, I am getting my longer fields truncated when I try and select... for example, here is a long GUID getting hacked '000083e0143381bbe8db4b0db7baa5...' Can I use DBD::CSV and have it read in long length fields?
  • Comment on DBD::CSV and long fields (LongReadLen not supported?)

Replies are listed 'Best First'.
Re: DBD::CSV and long fields (LongReadLen not supported?)
by Tanktalus (Canon) on Apr 26, 2011 at 22:45 UTC

    Can you give an example?

    use DBI; use strict; use warnings; my $csv = DBI->connect(qq'dbi:CSV:f_dir=.;csv_eol=\n') or die; my $d = $csv->selectall_arrayref(qq(select * from longread where a = ? +), {}, 1); print length $d->[0][1], "\n";
    I have a CSV file named "longread" with the first line looking like "A,B", and the second line like "1,..." where ... is your entire message copied in multiple times. This code is printing out "2072" - so it's handling a column that is 2072 characters long just fine... how long is your GUID?

      absolutely: here is the first 10 lines of text: http://pastebin.com/R8E6jVhs

      Here is the code:

      #! c:\program files\perl\bin use DBI; my $csv="viewmanifest.csv"; $dbh=DBI->connect('DBI:CSV:$csv') or die "Can't Connect"; $sth=$dbh->prepare("SELECT RowId, dahandle, darelativepath, dafilestat +e, dafilestatetime, dafilesource, fileguid, size, filemd5, child, chi +ldtype, childhandle, container, containerfolder, maghandle, mailconta +inererror, mailcontainer, mailfolder, mailfolderpath, contentmd5, cre +atedtime, datescanned, embeddedchildren, docclass, handle, docnum, os +folder, filename, docext, origdocext, filemode, fullpath, filetype, a +uxfiletype, lastaccesstime, lastmodifiedtime, lastchangetime, mimetyp +e, kftdesc, ocrpath, ocrstatus, ownergroup, owneruser, acls, parent, +parsingstatus, parserversion, altbody, importpath, importdevice, impo +rtarea, ancestordahandle, depotfile, inode, batch FROM $csv") or die +"Can't prepare"; $sth->execute(); $sth->dump_results(); $sth->finish(); $dbh->disconnect();

      Here are the results for the first line (notice the ellipses)

      '0', '000083e0143381bbe8db4b0db7baa5...', 'vmi/v920m3/weekly.20091014/pro...', 'imported', '2011-04-15-21-00-51', '000083e0143381bbe8db4b0db7baa5...', '', '0000000000000063906', 'a088be10e6f02f614385fa19628cfb6f', '', '', '', '', '', '', '', '', '', '', '', '', '2011-04-15-21-00-51', '', 'loose', 'a088be10e6f02f614385fa19628cfb...', '', 'vmi/v920m3/weekly.20091014/pro...', 'en_X11.xml', 'xml', '', '33188', '', 'extensible markup language (xml)', '', '2009-10-15-19-15-34', '2009-10-15-19-15-49', '2009-10-15-19-15-49', '', '', '', '', '100', '13936', '', '', '00000 SUCCESS', '3.4;3.4', '', '000083e0143381bbe8db4b0db7baa5...', '', '', '', '', '32457187', 'f8023f8fc92a4fc4'

      Other things of note, could have unicode filenames...

        dump_results is not meant to fetch data!

        use strict; use warnings; use autodie; use DBI; use Data::Peek; # For debugging only my $dbh = DBI->connect ("dbi:CSV:", undef, undef, { f_dir => ".", f_ext => ".csv/r", RaiseError => 1, PrintError => 1, }); my $sth = $dbh->prepare ("select * from viewmanifest"); my %rec; $sth->execute; $sth->bind_columns (\@rec{@{$sth->{NAME_lc}}}); while ($sth->fetch) { DDumper \%rec; # For debugging only print $rec{dahandle}, "\n"; } $dbh->disconnect;

        FWIW DBD::CSV doesn't need to support LongReadLen as the only limit to the size of the fields is implied by the available memory and the remaining space on disk (DBD::CSV is always AutoCommit so the data in the table should also fit on disk). The other database that I know of that ignores LongReadLen completely as it has direct bindings to the perl variable is DBD::Unify.


        Enjoy, Have FUN! H.Merijn