The CSV is what's produced by Thumbs Plus exporting the database. I can potentially massage it of course -- but the reason I'm reading it with Text::CSV was to *avoid* writing my own code to parse it. (it's a one-shot hack, it only has to work with *my particular* data, not in general, though, so hacking the input file is entirely on the table.) Taking off final commas is hard (something like 270,000 lines) but I can add content to the header to make it a "field" of no meaning. (And I'm only doing this because of repeated failures to get ODBC access to the database to work in anything except Libre Office -- which won't export it, and takes too long, three days and counting, to put it in a dialog box widget I might be able to cut and paste out of).

The error happens immediately, when I've only called the function to read the header line, so later lines shouldn't matter that I can see; and the time it takes doesn't suggest it's reading the whole file (270,000 lines of the size shown in my sample). Should I be making calls to define fields myself instead of reading the header, maybe? I'm just trying to do what seems the simple, direct way to use this code, if I'm guessing that wrong I'm open to change.

Adding auto_diag did not add anything at all to the output.

Versions are current:

----------------------------------------------------------------------
+---
        (no description)
        I/IS/ISHIGAKI/Text-CSV-1.95.tar.gz
        /usr/lib/perl5/site_perl/5.22/Text/CSV.pm
        Installed: 1.95
        CPAN:      1.95  up to date
        Kenichi Ishigaki (ISHIGAKI)
        ishigaki@cpan.org


ddb@DDB4 /cygdrive/p/work/tpdbfix/app
$ cpan -D Text::CSV_XS
Loading internal null logger. Install Log::Log4perl for logging messag
+es
Reading '/home/ddb/.cpan/Metadata'
  Database was generated on Wed, 07 Jun 2017 18:17:02 GMT
Text::CSV_XS
----------------------------------------------------------------------
+---
        (no description)
        H/HM/HMBRAND/Text-CSV_XS-1.29.tgz
        /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int/Text/C
+SV_XS.pm
        Installed: 1.29
        CPAN:      1.29  up to date
        H.Merijn Brand (HMBRAND)
        h.m.brand@xs4all.nl

[download]

In reply to Re^2: Text::CSV on Unicode file by dd-b
in thread Text::CSV on Unicode file by dd-b

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.