News from the trenches: a new attribute strict_eol will be available in the next release:

strict_eol

my $csv = Text::CSV_XS->new ({ strict_eol => 1 });
        $csv->strict_eol (0);
my $f = $csv->strict_eol;
[download]

If this attribute is set to 0, no EOL consistency checks are done.

If this attribute is set to 1, any row that parses with a EOL other than the EOL from the first row will cause a warning.
The error will be ignored and parsing continues. This warning is only thrown once. Note that in data with various different
line endings, \r\r will still throw an error that cannot be ignored.

If this attribute is set to 2 or higher, any row that parses with a EOL other than the EOL from the first row will cause
error 2016 to be thrown. The line being parsed to this error might not be stored in the result.

Any feedback welcome ...

The function (method) csv () will use the warning level by default.

There are still some edge-cases that might need a change, but the basics have settled.

Als always, you can get a pre-release from github.

1,A,Ape\r\n
2,B,Bear\r\n
3,C,Canary\n      <-- will warn with error 2016
[download]

Edge case example:

1,A,Ape\r\n
2,B,Bear\r\n
3,C,Canary\r\r\n  <-- will warn with error 2016 but also might not ret
+urn an empty row
[download]

The edge cases could also be considered "wrong" in all previous releases, and when strict_eol is disabled,
the behavior should be as before.

Older highlights since 1.49 ...

• Attribute skip_empty_rows extended
• Fix possible coredump in cache on non-IO parse
• Fix comment in last line
• Fix skip_empty_rows ("skip") and trailing newlines
• Fix comment_str with strict
• Strict won't hide previous error
• Fixes for strict under bind_columns
• Strict affected by column_names
• Add csv (on_error)
• Add XS line number in errors

Small updates

I now understand the fuzz people make about embedded newlines. Text::CSV_XS has always been able to deal with that (well, maybe not always, but at least for a long time already). The problem that people might have is reading the line in the perl script. Obviously,

  my $csv = Text::CSV_XS-New ({ binary => 1 });
  while (<>) {
      $csv->parse ($_);
      my @fields = $csv->fields ();
[download]

Will horribly fail, as <> will break too early.

The most recent snapshot now contains a t/45_eol.t, that tests all possible eol combinations. Have a look to see that the way to parse CSV with embedded newlines should be done similar to:

  use IO::Handle;
  my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
  while (my $row = $csv->getline (*ARGV)) {
      my @fields = @$row;
[download]

or, if you open files yourself, like:

  my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
  open my $io, "<", $file or die "$file: $!";
  while (my $row = $csv->getline ($io)) {
      my @fields = @$row;
[download]

I'm still thinking about the best way to add this to the docs.

Thanks very much for looking at this. I've tried 0.27, and it shows the same problem (see my original node, updated, for a test of sorts). I looked through your eol tests, and checked on exactly what was being written out to the temporary file in the \r cases. It appears that in those cases the file is terminated with a \r\n, rather than just \r. I think this may be why your tests pass but mine doesn't?

Cheers,

Tim

New snapshot just uploaded, in which eol => $/ is permitted for "\r". That extends successful parsing to line endings in the set undef, "\n", "\r\n", and "\r".


Enjoy, Have FUN! H.Merijn

And again, a lot of water has flown through <your favorite river>, so some highlights that might affect the end-users...

Personally I see myself going to use the stacked encoding quite a bit in the future:

 # Read GZipped CSV
 use PerlIO::gzip;
 my $aoa = csv (in => "test.csv.gz", encoding => ":gzip");

 # Write GZipped CSV
 use PerlIO::gzip;
 csv (
     in       => \@csv,
     out      => "test.csv.gz",
     encoding => ":via(gzip):encoding(utf-8)",
     );
[download]

1.40 - 2019-09-15

• Sheetnames are restricted to 31 characters in csv2xlsx
• The distributsion now has a cpanfile
• New: munge type db
• Do not generate xls/xlsx on empty CSV with csv2xlsx and csv2xls
• New: support $csv->formula (sub { ... })
• New: Support stacked encodings

1.39 - 2019-03-15

• Add options -w/-b/-Z to csvdiff
• Fix strict on streaming EOF

1.38 - 2018-12-30

• Name the duplicate headers on error 1013
• Add missing attributes to default list (doc only, David H. Gutteridge)
• Add support for combined keys
• Look at $NO_COLOR for csvdiff
• Add support for key-value pair

1.37 - 2018-09-27

• Add munge as alias for munge_column_names

1.36 - 2018-06-26

• Fixed memory leak (Thanks DaveM)
• Add undef_str attribute
• Move from DynaLoader to XSLoader

I finally was able to strike the EBCDIC TODO-item. After Karl and I got perl-5.32.0 built on z/OS EBCDIC native with dynamic loading and threads, the next step was to get Text::CSV_XS passing all tests on it. We need a goal, and this one was a real challenge, but it now works, out of the box! (I updated the TODO list in the OP)

I just released 1.44, which was also tested with the new (not set in stone) defaults that the core development team envisions for perl-7, and everything passes \o/

Check out the updated tools: csv-check and csv2xlsx.

1.44 - 2020-07-22

• EBCDIC now fully functional
• Prevent false negative CPANTESTERS fail reports
• Fully tested on perl-5.32.0 and perl-5.33.0
• Fix partly decoded fields in header (found by Eric Brine)

1.43 - 2020-05-27

• Add --parser-opt=value to csv-check
• Add -L & -X to csv-check
• Fix undef value in attribute setters for strings (issue 24)
• Document quote (undef) is not supported

1.42 - 2020-05-19

• Update to Devel::PPPort-3.58
• Unicode fixes for csv2xls and csv2xlsx
• Add internal buffers to cache diagnostics
• Fix positional reporting in examples/csv-check
• Allow passing CSV parsing attributes to csv-check
• Proof reading - doc fixes by Klaus Baldermann <soonix> (PR#21)
• Fix type caching (RT#132344)
• Small doc fix by Nick Tonkin <1nickt> (PR#22)
• Fix sep=; being ignored in ->header (issue 23)

1.41 - 2020-02-16

• Update to Devel::PPPort-3.56
• csv2xls uses sheetname as csv2xlsx
• csv2xlsx: support images (each image gets its own tab)
• More docs. (Data validation)
• No binary literals in fixed error messages
• Fix auto_diag > 2 to die when headers are used (issue 19)

Thanks for the links. I'll scan them.

Meanwhile Text::CSV_XS 0.27 has left the building and should be available to all when CPAN synced your favourite mirror.

2007-05-31  0.27 - H.Merijn Brand

        * checked with perlcritic (still works under 5.00504)
          so 3-arg open cannot be used (except in the docs)
        * 3-arg open in docs too
        * Added a lot to the TODO list
        * Some more info on using escape character (jZed)
        * Mention Text::CSV_PP in README
        * Added t/45_eol.t, eol tests
        * Added a section about embedded newlines in the pod
        * Allow \r as eol ($/) for parsing
        * More docs for eol
        * More eol = \r fixes, tfrayner's test case added to t/45_eol.
+t
[download]

Work already progresses for the next snapshot that is to include ways to allow buggy formats

2007-06-01  0.28 - H.Merijn Brand

        * Added allow_loose_quotes  (see doc)
        * Added t/65_allow.t
        * Added allow_loose_escapes (see doc) RT 15076

=item allow_loose_quotes

By default, parsing fields that have C<quote_char> characters inside
an unquoted field, like

 1,foo "bar" baz,42

would result in a parse error. Though it is still bad practice to
allow this format, we cannot help there are some vendors that make
their applications spit out lines styled like this.

=item allow_loose_escapes

By default, parsing fields that have C<escapee_char> characters that
escape characters that do not need to be escaped, like:

 my $csv = Text::CSV_XS->new ({ esc_char => "\\" });
 $csv->parse (qq{1,"my bar\'s",baz,42});

would result in a parse error. Though it is still bad practice to
allow this format, this option enables you to treat all escape charact
+er
sequences equal.
[download]

I'm playing to see if allow_whitespace is also feasable

• I would like to find a good logic to parse the data
• Multiple DB Connection Problem
• Getting columns from csv file in perl.

I'm sure there's lots more.

Text::CSV_XS should be in the standard Perl distribution!!!

I think that making the eol option actually honour $/ would be extremely useful, and I think your suggested API for that would work fine. As I understand it (and the current behaviour of Text::CSV_XS certainly seems to bear me out) the module only handles \015\012 and \012 as CSV line endings for parsing input. It would be tremendously helpful if \015 was also supported. I'm assuming it's not, since I've never been able to get it to work despite occasional attempts over the years. There's also a note in the POD (CAVEATS) suggesting that alternate line endings are unsupported. I'm often surprised at the number of people out there still using spreadsheet software that generates CSV files with \015 line endings. I think older Mac OS machines, and some ill-behaved Mac OSX applications are at fault (if fault it really is).

Cheers,

Tim

Update: based on the principle that every bug report should at least take a stab at giving a test case, here's what I'm talking about:

perl -MText::CSV_XS -MIO::File -e '$/="\r"; $f = IO::File->new_tmpfile
+; print $f "a,b,c$/"; seek($f,0,0); $c = Text::CSV_XS->new({eol => $/
+}); print join("|",@{ $c->getline($f) })'
[download]

A test case calls for proof :)

use strict;
use warnings;

use Text::CSV_XS;

$/ = "\r";

print "Testing with version $Text::CSV_XS::VERSION\n";
my $csv = Text::CSV_XS->new ({ eol => $/ });
my $str = "a,b,c$/";
open my $fh, "<", \$str or die "ScalarIO: $!\n";
if (my $row = $csv->getline ($fh)) {
    print join "|", @$row, "\n";
    }
else {
    $csv->error_diag;
    }
[download]

Generates:

$ perl eoltest.pl
Testing with version 0.58
a|b|c|
[download]


Enjoy, Have FUN! H.Merijn

Thanks to all the feedback from all the different corners of the perl community, I'm doing far more than I planned :)

Feel free to comment, request features, or give feedback on whatever part of this module you want. Meanwhile, 0.30 will leave for the wild wide world soon ...

2007-06-18  0.30 - H.Merijn Brand

        * ,\rx, is definitely an error without binary (used to HANG!)
        * Fixed bug in attribute caching for undefined eol
        * Cleaned up some code after -W*** warnings
        * Added chomp_verbatim. HIGHLY EXPERIMENTAL!
        * More test to cover the really dark corners and edge cases
        * Even more typo fixes in the docs
        * Added error_diag ()
        * Added t/80_diag.t
        * Added DIAGNOSTICS section to pod

2007-06-08  0.29 - H.Merijn Brand

        * Removed an unused 'use Data::Dumper'
        * Added $csv->eof () RT 27424
        * Two typo's in the doc's (Amsterdam.pm)
        * Modified examples/speed.pl to better show the diffs between 
+versions
        * Cache attribute settings and regain speed of 0.23! and beyon
+d
          Relative overall speeds (YMMV, use examples/speed.pl to chec
+k),
          the 1.0x versions are from Text::CSV_PP.

                        0.23 0.25 0.26 0.27 0.28 0.29 1.00 1.02 1.05
                        ==== ==== ==== ==== ==== ==== ==== ==== ====
            combine   1   62   61   61   60   58  100   14   14   14
            combine  10   41   41   41   42   42  100    6    6    6
            combine 100   35   35   36   36   36  100    5    5    5
            parse     1  100   92   87   90   81   96   19   19   17
            parse    10   95  100   86   97   94   94   15   16   14
            parse   100   90   95   84   94   92  100   16   16   14
            print    io   93   89   91   91   95  100    0    0    6
            getline  io   90   92   84   87   89  100    0    0   13
                        ---- ---- ---- ---- ---- ---- ---- ---- ----
            average       75   75   71   74   73   98    9    9   11

        * Removed prototypes
        * Added a SPECIFICATION section to the pod
        * Safe caching of eol, and a warning in the docs
        * Rewrote t/20_file.t do do actual file IO instead of IO_Scala
+r fake
        * Small optimization for parse (juerd)
        * Added make target test_speed
        * Merged the items from CAVEAT to SPECIFICATION
        * Updated the TODO and Release Plan sections
        * Speed up internals by using references instead of copies (ju
+erd)
        * Reworked error_input, which also had an undetected internal 
+error
        * Added IO tests for lexical IO handles and perlio IO to/from 
+scalars
[download]

The release candidate is up for you to test.

A lot of water flowed through many rivers and a lot of tests have been executed since 1.10. I'll show the highlights ...

My reason to join PerlMonks many eons ago was to get a grasp on what CSV problems were encountered by users of CSV parsers, and I am so happy with all the feedback I got both here and in the ChatterBox.

THANK YOU ALL!

The git repo now has a documentation section with preformatted HTML en MarkDown.

1.35 - 2018-03-21

• Make detect_bom result available
• Add csv (out => \"skip") - suppress output deliberately
• Allow sub as top-level filter

1.34 - 2017-11-05

• Bad arg for formula (like "craok") will now die with error 1500
• Add a prominent section about BOM handling
• Make sheet label more portable (csv2xlsx)
• Allow munge => \%hash
• Preserve first row in csv (set_column_names => 0)

1.33 - 2017-10-19

• Add support for formula actions (issue 11)

1.32 - 2017-09-15

• Add keep_headers attribute to csv ()
• Fail on invalid arguments to csv
• Fix header method on EOL = CR (RT#122764)

1.31 - 2017-06-13

• Fix already decoded BOM in headers
• New options in csv-check
• "escape" is alias for "escape_char" for consistency.
• Improve csv-check auto-sep-detection

1.30 - 2017-06-08

• Disable escape_null for undefined escape_char
• Fix ->say for bound columns (RT#121576)

1.29 - 2017-04-25

• New attribute "strict" (also addresses RT#121350)

1.28 - 2017-03-22

• Fix length problem with bound empty fields and UTF-8 (RT#120655)

1.27 - 2017-03-02

• Attribute sep/sep_char is not allowed to be undefined
• Increased test coverage: added errors 1008 and 1014
• Default for escape_null in csv () is now false
• New error code for illegal argument(s)/parameter(s) (RT#119827)
• Fix tests for perl without dot in @INC
• Fix crlf issue for csv () on Windows (RT#120466)

1.26 - 2016-11-29

• is_missing (0) on empty line returns 1 for keep_meta_info = true

1.25 - 2016-08-28

• Allow lc, uc, and coderef for csv () headers attribute
• Document for eof when the last line has an error (RT#115954)
• Allow csv () to call header () with all supported arguments

1.24 - 2016-07-09

• Set auto-wrap on for csv2xls with embedded newlines
• Add examples/csv2xlsx, the MSExcel-2007+ version of csv2xls
  Includes new feature to merge multiple CSV's into a single xlsx
• Fix parse error in complex option combo (RT#115953)

1.23 - 2016-03-25

• Skip unsupported encodings
• Fix crash on error in parsing with bound columns (RT#113279)
• Add predefined filters (not_blank, not_empty, filled)

1.22 - 2016-02-23

• Add new method header

1.21 - 2016-01-06

• Alias %_ to row in hashref mode for csv function attributes on_in and before_out
• Fix \r\n ending with allow_loose_escapes (reported by Bas Bloemsaat)

1.20 - 2015-10-09

• Add known_attributes function/method
• Add contributor notes
• Allow undef as value for aliased attributes

1.19 - 2015-06-17

• Guard tests against $PERL_UNICODE
• Numeric options were sometimes interpreted as boolean
• Safer meta_info use

1.18 - 2015-05-22

• Add quote_empty attribute
• Add database NULL documentation
• Inherit csv attributes in csv () when called in void context
• micro-optimisation for combine/print (empty fields will be faster)

1.17 - 2015-04-24

• Enable overruling $csv in csv ()
• Allow encoding to be shortened to enc in csv ()
• Allow filter to alter content
• Add say (print with default eol => $\)
• Allow MS sep=; on first line (RT#100304)

1.16 - 2015-03-02

• filter made more useful (access to other fields)

1.15 - 2015-02-11

• Remove perl recommendation from META as it breaks cpan clients

1.14 - 2015-02-02

• Move to github
• Add csv (filter => {});

1.13 - 2015-01-03

• Fix quote_binary (#RT100676)
• Fix csv () for hashrefs with aliased headers

1.12 - 2014-11-01

• Add field number to error_diag
• Fixed non-IO parsing multi-byte EOL
• Fixed a possible missed multi-byte EOL
• Allow hashref for csv ()'s headers attribute
• Allow encoding on all output handles in csv ()
• Include doc changes as ticketed in the Text::CSV queue
• Fix parallel testing issue
• Allow csv as method call (not using the object)
• Rename quote_null to escape_null
• Give meaning to keep_meta_info on output (RT#99941)

1.11 - 2014-08-16

• Fixed eof (RT#97742)
• Check for attribute conflicts early
• csv (in => ..) now defaults to *STDOUT for out
• Support for multi-byte quote_char
• New attribute "key" for csv ()

I think my recent changes to csv2xlsx warrants an update.

As this is on github but not yet released, feel free to comment on the new -S/--split feature, so I can update the docs or amend the code.

To explain the new options: I was faced with a 50_000 line CSV file that had logical vertical sections. The section start was clearly marked. Say you have a CSV file with orders per customer, grouped by customer, and every such section starts with Customer,34526,"Foo Tool Factory",20220601,... followed by the orders.

What I wanted was to convert this CSV to Excel (as required by the other end) where each customer had its own sheet. With the new options that would be:

$ csv2xlsx -S A=Customer --sl=C orders.csv
[download]

Start a new sheet when column A is "Customer" and then use the value in column C for the matching record as label for the sheet.

-S (or --split) also supports syntax for regex/empty/undef/compare and can be stacked.

Here are the highlights of changes since my previous posts:

1.49 - 2022-06-21

• csv2xlsx --split=CxP [--split-label=C]
• Full documentation/manual for csv2xlsx

1.48 - 2022-05-24

• kh => 1 (use internal headers)
• Add constants for META flags (RT#142508)
• Export :CONSTANTS
• Fix sep_set typo (issue 37)

1.47 - 2021-12-21

• Make error 2014 catchable
• Suppress empty lines on empty input (issue 34)

1.46 - 2021-03-24

• New attribute comment_str (RFC 4180-bis)
• New attribute skip_empty_rows (RFC 4180-bis)
• http -> https in links in docs
• Fix several issues with auto-detecting \r as EOL

1.45 - 2020-12-24

• Allow adding to existing data in csv (out => $ref)
• examples/csv2xlsx : -L # to limit records to export
• examples/csv2xlsx : --de # for possible double-encoding fix
• Add examples/csv-split
• examples/csv2xlsx : Check if .xlsx is creatable before conversion

This thread++ lead to–

You have 1 vote left today

:P

This is likely the release that took me longest to test, but it now passes under 80 versions of perl ranging from 5.6.1 to 5.21.1 on OS's I had (indirect) access to: HP-UX 10.20, 11.00, 11.11, 11.23pa, 11.23ipf, 11.31, AIX-5.3, Win7, MacOSX, NetBSD, cygwin and of course Linux 32bit and 64bit.

Version 1.10 supports multi-byte separation characters, which means that you now can use Unicode characters like \N{INVISIBLE SEPARATOR}, \N{FULLWIDTH COMMA} or even \N{SNOWMAN}.

This release opens development for also supporting multi-byte quotation. More on that later.

1.10 - 2014-08-02

• Support for scalar ref in out: csv (out => \(my $x = ""), ...)
• Support for multi-byte sep_char
• Simplified the cache coding

1.09 - 2014-06-09

• Documentation updates
• Allow disjointed CSV cell fragment lists

I want to thank all of those that provided feedback to my questions. Enjoy!

2007-10-24  0.32 - H.Merijn Brand
 
        * Added $csv->error_diag () to SYNOPSIS
        * Added need for diag when new () fails to TODO
        * Fixed a sneaked-in defined or in examples/csv2xls
        * Plugged a 32byte memory leak in the cache code (valgrind++)
        * Some perlcritic level1 changes
 
2007-07-23  0.31 - H.Merijn Brand
 
        * Removed prototypes in examples/csv2xls
        * Improved usage for examples/csv2xls (GetOpt::Long now does -
+-help/-?)
        * Extended examples/csv2xls to deal with Unicode (-u)
        * Serious bug in Text::CSV_XS::NV () type setting, causing the
          resulting field to be truncated to IV
[download]

And a new speed comparison table. 0.23 .. 0.32 are Text::CSV_XS, and 1.00 .. 1.06 are Text::CSV_PP. I have no idea why 1.06 is not yet on CPAN.

            0.23 0.25 0.26 0.27 0.28 0.29 0.30 0.31 0.32 1.00 1.02 1.0
+5 1.06
            ==== ==== ==== ==== ==== ==== ==== ==== ==== ==== ==== ===
+= ====
combine   1   60   60   60   59   56  100   98   93   96   14   14   1
+4   14
combine  10   40   40   41   41   40  100   96   92   95    6    6    
+6    6
combine 100   34   33   34   33   34  100   92   89   90    5    5    
+5    5
parse     1  100   86   92   89   86   96   98   93   98   20   19   1
+7    8
parse    10   94   85   98   98  100   96   92   87   94   16   16   1
+4    5
parse   100   88   81   93   90   97  100   97   93   99   16   16   1
+4    5
print    io   92   90   91   91   92  100   95   94   94   68   66    
+6    6
getline  io   88   80   90   88   90  100   98   93   98    -    -   1
+3    5
            ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---
+- ----
average       74   69   74   73   74   99   95   91   95   18   17   1
+1    6
[download]


Enjoy, Have FUN! H.Merijn

  file: $CPAN/authors/id/H/HM/HMBRAND/Text-CSV_XS-0.42.tgz
  size: 86136 bytes
   md5: 1cf4491f48965793f1e31fc74159f20f
 
We can do MAGIC now!
 
Dumping the content of a database ($dbh) table ($tbl) to CSV:
 
  my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
  open my $fh, ">", "$tbl.csv" or die "$tbl.csv: $!";
  my $sth = $dbh->prepare ("select * from $tbl");
  $sth->execute;
  $csv->print ($fh, $sth->{NAME_lc});
  while (my $row = $sth->fetch) {
      $csv->print ($fh, $row);
      }
  close $fh;
 
2008-04-16  0.42 - H.Merijn Brand
 
        * Generate META.yml myself. I won't use Build.PL
        * Array-refs now accept scalars with magic:
          $csv->print (*STDOUT, $sth->{NAME_lc});
        * More/better examples
        * Added t/76_magic.t
[download]

2008-04-21  0.43 - H.Merijn Brand

        * parse errors try to remember failing position
        * used valgrind to test for leaks (devel-only)
        * used Test::Valgrind as alternative leak check (devel-only)
        * improve documentation for error 2023
        * nailed the loose quotes in quoted fields
[download]

The upcoming 0.44 will have:

        * Fixed the error position returned as third arg in error_diag
+ ()
        * Made examples/csv-check use this even more vebose
        * Removed double-double quote from TODO
[download]

2008-10-30 0.58 - H.Merijn Brand

        * Small typo in test message (RT#40410, JPL)
        * Parse error test on "\0 (RT#40507)
        * Fixed allow_loose_escapes bug disclosed by RT#40507

2008-10-21 0.57 - H.Merijn Brand

        * Don't bootstrap twice. Don't know how/when it came in there
[download]

With the bugfix in 0.58, the Devel::Cover coverage (including XS) went up to 99.8%!

---------------------------- ------ ------ ------ ------ ------ ------
+ ------
File                           stmt   bran   cond    sub    pod   time
+  total
---------------------------- ------ ------ ------ ------ ------ ------
+ ------
CSV_XS.xs                      99.6    n/a    n/a    n/a    n/a    n/a
+   99.6
blib/lib/Text/CSV_XS.pm       100.0  100.0  100.0  100.0  100.0  100.0
+  100.0
Total                          99.7  100.0  100.0  100.0  100.0  100.0
+   99.8
---------------------------- ------ ------ ------ ------ ------ ------
+ ------
[download]

2008-03-01  0.35 (Valloire) - H.Merijn Brand

        * use warnings/$^W = 1
        * Tested on 5.10.0, 5.8.8, 5.6.2, and 5.005_04, Strawberry and
+ Cygwin
        * Diagnostics for failed new ()
        * New 'blank_is_undef' option
        * Updated docs
        * Extended the DIAGNOSTICS a bit
        * Updated TODO
        * Fixed allow_whitespace issue, revealed by blank_is_undef
        * Re-enabled some tests
        * Fixed parse error that passed for q{1, "bar",2} with escape_
+char +
        * Reversed an erroneous test result in the funny combo section
        * Extended diagnostics tests
        * Extended XS coverage
        * Removed error 2033
[download]

  file: $CPAN/authors/id/H/HM/HMBRAND/Text-CSV_XS-0.40.tgz
  size: 85057 bytes
   md5: cb8b2af20925b832159f34eed9793666

2008-04-07  0.40 - H.Merijn Brand

        * Implemented getline_hr () and column_names () RT 34474
          (suggestions accepted from Mark Stosberg)
        * Corrected misspelled variable names in XS
        * Functions are now =head2 type doc entries (Mark Stosberg)
        * Make SetDiag() available to the perl level, so errors can
          be centralized and consistent
        * Integrate the non-XS errors into XS
        * Add t/75_hashref.t
        * Testcase for error 2023 (Michael P Randall)
        * Completely refactored the XS part of parse/getline, which
          is now up to 6% faster. YMMV
        * Completed bind_columns. On straight fetches now up to three
          times as fast as normal fetches (both using getline ())

       getline_hr

       The "getline_hr ()" and "column_names ()" methods work together
+ to
       allow you to have rows returned as hashrefs. You must call
       "column_names ()" first to declare your column names.

        $csv->column_names (qw( code name price description ));
        $hr = $csv->getline_hr ($io);
        print "Price for $hr->{name} is $hr->{price} EUR\n";

       "getline_hr ()" will croak if called before "column_names ()".

       column_names

       Set the keys that will be used in the "getline_hr ()" calls. If
+ no keys
       (column names) are passed, it'll return the current setting.

       "column_names ()" accepts a list of scalars (the column names) 
+or a
       single array_ref, so you can pass "getline ()"

         $csv->column_names ($csv->getline ($io));

       "column_names ()" croaks on invalid arguments.

       bind_columns

       Takes a list of references to scalars (max 255) to store the fi
+elds
       fetched "by getline_hr ()" in. When you don't pass enough refer
+ences to
       store the fetched fields in, "getline ()" will fail. If you pas
+s more
       than there are fields to return, the remaining references are l
+eft
       untouched.

         $csv->bind_columns (\$code, \$name, \$price, \$description);
         while ($csv->getline ()) {
             print "The price of a $name is \x{20ac} $price\n";
             }
[download]

I don't know what the source of the problem is, but I'm posting to note that the Issues link from the CPAN page is broken. I was impressed to see that there was only one issue, and wanted to check out what it might be.

And also, I use Text::CCSV in my current work, and it's awesome to use. I had been using just

  print { $csv_files{$table} } join( ',', @{ $data{$table} } ) . "\n";
[download]

  $csv_obj->print ( $csv_files{$table}, $data{$table} );
[download]

• Issues was fixed in this commit a month ago, which will be part of the next release.
• The corrected issue link now points to just one single issue (about the issue link, which has been resolved)
• The RT site is unstable. It sometimes shows ticket 119312 about threading, which stalled.
• I don't know about Text::CCSV. I know about App::CCSV and Text::CSV.

If you think the commits on Text::CSV_XS warrent a release, I'll consider.


Enjoy, Have FUN! H.Merijn

Thanks for these updates! I just happened to visit your profile, saw your ask to visit an old node (as one does), and posted an update. I believe the XS module gets automatically pulled in by the main module if the environment warrants. Whatever's actually being used, the module is terrific -- I've used it on and off since 2016, and probably before that as well.

Awesome work. Thanks again.

Alex / talexb / Toronto

As of June 2025, Groklaw is back! This site was a really valuable resource in the now ancient fight between SCO and Linux. As it turned out, SCO was all hat and no cattle.Thanks to PJ for all her work, we owe her so much. RIP -- 2003 to 2013.

It is very good to see people are actually using error_diag (), but it also means people find bugs.

2008-09-01 0.53 - H.Merijn Brand

        * SvUPGRADE is a safer choice than sv_upgrade (Lubomir Rintel,
+ RT#38890)
        * bring docs in sync with reality for msg 2023
        * Default eol for print is $\
        * examples/csv-check should default to CSV , not to ;
        * Tests for SetDiag (0)
        * Tests for error 2030
        * Code cleanup (Devel::Cover++)

2008-06-28  0.52 - H.Merijn Brand

        * Using undef for hash keys is a bad plan
        * Fix, tests, and documentation for column_names ()
[download]

I finally scratched a longstanding personal itch. No more need for 'useIO::Handle;'. Text::CSV_XS will now do that for you automatically at the moment it is needed, and only once, so no performance penalty. This will simplify one-liners.

2008-03-06  0.36 - H.Merijn Brand

        * Updated ppport.h
        * auto-load IO::Handle when needed
        * Tested with all available perl builds, including a fresh
          threaded 5.11 (blead)
[download]

2008-03-11  0.37 - H.Merijn Brand

        * Copied GIT repo to public mirror
        * Fix leak / make meta info available to getline () + tests
[download]

    Recent changes can be (re)viewed in the public GIT repository at
    git://repo.or.cz/Text-CSV_XS.git
[download]

Mac users rejoyce!

I've come to the point where I now really think that for normal CSV, you won't have to set the eol attribute anymore for parsing CSV data. All off \n, \r\n, and now also \r are automatically recognized in streams.

2009-10-10 0.69 - H.Merijn Brand

• Missing end quotes in error code docs
• examples/csv-check now shows detected eol
• Auto detection of eol => "\r" in streams
• Optimized caching. All cache changes now in XS

2009-10-04 0.68 - H.Merijn Brand

• Attribute auto_diag now localizes to +1 if autodie is active
• Output name generation in csv2xls (RT#48954)
• Added examples/csvdiff to examples/
• Reordered docs. Rewrote SYNOPSIS to be more like a real-world code example

2009-08-08 0.67 - H.Merijn Brand

• Fix empty_diag typo for attribute handler
• Fix AUTOMATED_TESTING erroneous skip

  file: $CPAN/authors/id/H/HM/HMBRAND/Text-CSV_XS-0.41.tgz
  size: 85570 bytes
   md5: f704fb8ad057a36e7cc0fb892c0f940a

2008-04-11  0.41 - H.Merijn Brand

        * error_diag () subclassable
        * typo in bind_columns () docs
        * examples/csv2xls now uses getline ()
        * better test for getline in t/75_hashref.t (makamata)
        * document return value of getline () with bind_columns ()
        * add perl version prereq to META.yml
[download]

2008-06-17  0.51 - H.Merijn Brand

        * Allow UTF8 even without binary => 1
        * Fixed a few pod typo's
        * Lifted the max of 255 for bind_columns

2008-06-04  0.50 - H.Merijn Brand

        * Skip a few tests in automated testing, as they confuse
          reports. This is important for the automated sites that
          mark modules as fail if it is not an obvious full PASS
        * 0.46 caused the last still open RT bug to be closed!
        * Tested on 5.11.0, 5.10.0, 5.8.8, 5.6.2, and 5.005_04,
          Strawberry and Cygwin

2008-06-04  0.46 - H.Merijn Brand

        * In examples add die on failed close calls
        * Use Test::MinimumVersion (not distributed)
        * Added option -F to examples/csv2xls
        * More source code cleanup
        * Nailed the UTF8 issues for parsing
        * Nailed the UTF8 issues for combining

2008-04-23  0.45 - H.Merijn Brand

        * Forgot to pack examples/parser-xs.pl

2008-04-23  0.44 - H.Merijn Brand

        * Fixed the error position returned as third arg in error_diag
+ ()
        * Made examples/csv-check use this even more vebose
        * Removed double-double quote from TODO
        * Added examples/parse-xs.pl (attempt to fix bad CSV)
[download]

Here's my X-Mas present:

0.80 - 2010-12-24

• Implement getline_all () and getaline_hr_all ()
• Fixed another parsing for eol = \r (RT#61525)

0.79 - 2010-11-26

• Use correct type for STRLEN (HP-UX/PA-RISC/32)
• More code coverage
• EOF unreliable when line-end missing at eof

0.78 - 2010-11-26

• Version 0.77 broke MacOS exported CSV files with only \r

The two new methods:

• getline_all

     $arrayref = $csv->getline_all ($io);
     $arrayref = $csv->getline_all ($io, $offset);
     $arrayref = $csv->getline_all ($io, $offset, $length);
  [download]

  This will return a reference to a list of getline ($io) results. In this call, keep_meta_info is disabled. If $offset is negative, as with splice (), only the last abs ($offset) records of $io are taken into consideration.

  Given a CSV file with 10 lines:

     lines call
     ----- ---------------------------------------------------------
     0..9  $csv->getline_all ($io)         # all
     0..9  $csv->getline_all ($io,  0)     # all
     8..9  $csv->getline_all ($io,  8)     # start at 8
     -     $csv->getline_all ($io,  0,  0) # start at 0 first 0 rows
     0..4  $csv->getline_all ($io,  0,  5) # start at 0 first 5 rows
     4..5  $csv->getline_all ($io,  4,  2) # start at 4 first 2 rows
     8..9  $csv->getline_all ($io, -2)     # last 2 rows
     6..7  $csv->getline_all ($io, -4,  2) # first 2 of last  4 rows
  [download]

• getline_hr_all

     $arrayref = $csv->getline_hr_all ($io);
     $arrayref = $csv->getline_hr_all ($io, $offset);
     $arrayref = $csv->getline_hr_all ($io, $offset, $length);
  [download]

  This will return a reference to a list of getline_hr ($io) results. In this call, keep_meta_info is disabled.

Firstly, what a great module, thanks! I've always found it reliable and effective.

It would be handy if string() returned the string read by getline().

Although this does seem to work...

use Text::CSV_XS;
use IO::File; # tell/seek fails if i omit this line

my $file = "1,2,3\n4,5,6\n7,8,9\n10,11,12\n13,14,15";
open (my $io, '<', \$file); # Open filehandle to in memory file
my $csv = Text::CSV_XS->new();

while (1) {

  my $start = tell($io);
  my $colref = $csv->getline($io);
  last unless $colref;

  print "Array: @$colref\n";

  my $length = tell($io) - $start;

  seek($io, $start, 0);
  read($io, my $string, $length);

  print "String: <$string>\n";

}
[download]

What do you think? Would it be possible for Text::CSV_XS to return the string more efficiently?

I've also wondered whether eol would always be limited to a finite set of predefined strings?

Thanks again.
Steve

That could be done, but not by default, as this has a big impact on performance, as now every getline () result has to be copied/stored in a location that string () can return to the user.

The solution you post is only viable if the input stream is seek'able, and will not work on pipes. This is one of the reasons why this module is not capable of determining the line characteristics on the fly.


Enjoy, Have FUN! H.Merijn

You might thank Lady_Aleena for my motivation, but the new csv () function has reached a state that pleases most people I talked to, but still having the possibilities to go wild. Read the docs

1.08 - 2014-05-17

• Documentation updates
• Allow disjointed CSV cell fragment lists

1.07 - 2014-04-28

• Allow ref to scalar for csv (in => ...)
• Allow CODE ref to in attribute
• Allow * in fragment cellspec's bottom-right cell

1.06 - 2014-04-20

• Fix possible fail in tests on Windows (Thanks Mithaldu for explaing)
• Only close file handles in csv () for files
• new callbacks for csv ()

1.05 - 2014-03-02

• Allow case insensitive attributes and attribute aliases (quote_always = always_quote)
• Enhanced the csv () function (diagnostics)
• Start callbacks support
• Minor doc fixes
• Make subclassing safer

1.04 - 2014-02-06

• column_names () with no argument now returns the empty list instead of undef when no columns defined
• fragments (rfc7111) now also support AoH (was AoA only)
• Error code conflict for fragments resolved to 2013
• New function "csv" (not a method)

I know some rows will be too short. I just want to skip them or log them and move on. Or maybe there's a a way I can "pre-test" them before calling getline()?

Text::CSV_XS comes with examples. There is csv-check and parser-xs.pl. Both show ways to parse (broken) CSV files reliably. Take the latter as an example.

Making fatal errors recoverable is very hard, as the module does not buffer. Once the stream has been read, it is hard to backup in it. You can still probably use $csv->error_input () after a caught eval'ed failure.


Enjoy, Have FUN! H.Merijn

2008-10-15 0.55 - H.Merijn Brand

        * Improve documentation on eol
        * Unicode on perl-5.8.[0-2] sucks. Don't use it!
        * Test error codes in expected IO failures
        * Allow SetDiag to be used as class method
        * Document the MS/Excel separation character
        * Hint that eof is not an error per se (RT#40047)

2008-09-04 0.54 - H.Merijn Brand

        * IO failure in print () was not propagated (ilmari, RT#38960)
[download]

2009-01-23 0.59 - H.Merijn Brand

        * Wrong e-mail in META.yml
        * Missing $IO argument in bind_columns example (docs only)
        * Upped Copyright notices to 2009
        * Added warning for parse () (RT#42261)
        * Small optimisations (Goro Fuji, RT#42517)
        * ppport.h updated to 3.15
        * Added git clone command to README
        * ppport.h updated to 3.16-pre
        * Optimize getline/print method calls (Goro Fuji, RT#42517)
        * Decode *_char attributes for perl 5.8.2 and up
[download]

Three more releases later. People still make me find hidden features/bugs :)

2009-04-03 0.64 - H.Merijn Brand

        * Skip perlio tests for perl older than 5.8, as perlio
          was experimental in 5.6
        * Up Devel::PPPort to 3.17
        * Fix initialisation of eol => undef (could cause core dump)
        * Added configure_require to META.yml

2009-03-20 0.63 - H.Merijn Brand

        * Fixed allow_whitespace with sep_char = TAB (RT#44402)

2009-03-13 0.62 - H.Merijn Brand

        * Prevent warnings in older perls (without utf8)
        * All known errors are covered and/or documented. TODO dropped
        * Unicode TODO dropped. All covered, or use Text::CSV::Encoded
        * Documented the examples
[download]

2009-05-14 0.65 - H.Merijn Brand

    * Initial known errors can now be checked on number (1002)
    * More tests for illegal combinations
    * Added -u option to examples/csv-check to validate utf-8 encoding
    * Correct documentation for error_diag () return value in case of
      constructor failure (Slaven, RT#46076)
    * All error_diag () returns should now be dual-var (both numeric
      and string context valid)
    * Remove (3) from L<..> links (Slaven, RT#46078)
[download]

2010-05-03 0.73 - H.Merijn Brand

• Improve date conversion in examples/csv2xls
  new option -D allows column selection for date conversions
• Tested under perl-5.12.0 (and 21 other versions of perl)
• Added a note about EBCDIC data files
• Test suite is now safe for parallel test (prove --shuffle -j6)

Together with the 1.611 release of DBI, it made a new release possible for DBD::CSV

2010-05-03 DBD::CSV-0.29

Some highlights from over the past few releases:

• Support for f_encoding and f_lock
• Documentation updates
• Adjustments for windows
• Mark all non-\w chars illegal in field and table names
• Fix field types after execute
• Fix for NULL joins

Which leads you to:

           $dbh = DBI->connect ("dbi:CSV:", undef, undef, {
               f_schema         => undef,
               f_dir            => "data",
               f_ext            => ".csv/r",
               f_lock           => 2,
               f_encoding       => "utf8",

               csv_null         => 1,

               RaiseError       => 1,
               PrintError       => 1,
               FetchHashKeyName => "NAME_lc",
               }) or die $DBI::errstr;
[download]

2008-10-21 0.56 - H.Merijn Brand

        * Update to ppport.h 3.14_01
        * Updated docs (Unicode, TODO, Release Plan)
        * As Text::CSV::Encoded is about to be released, refer to it
          in the documentation
        * Default for eol is "", undef now treated as ""
        * Don't print $\ twice (eol prevails over $\ in ->print ())
          Fix only works in perl5.8 and up
        * Undef treated as 0 for boolean attributes
        * Trailing whitespace in pod removed
        * Sync up doc with latest Text::CSV::Encoded
        * YAML declared 1.4 (META.yml) instead of 1.1 (YAML)
[download]

2009-03-08 0.61 - H.Merijn Brand

        * valgrind found a possible uninitialized value
        * Restriction in print () was only for really old perls
        * Fix for bind_columns () initialisation (vincent, RT#43927)

2009-01-27 0.60 - H.Merijn Brand

        * Optimize for threaded perls. (Goro Fuji, RT#42517)
          Non-threaded perls won't notice
[download]

2010-09-29 0.74 - H.Merijn Brand

• Spelling fixes
• Real eol support for parsing streams (beyond \n, \r and \r\n)
• Clarify doc for always_quote to not quote undef fields
• Clarify UTF8 process for print () and combine ()

This release passed 35 versions of perl on a single box using a slightly modified version of Module::Release :)

2010-10-09 0.76 - H.Merijn Brand

• Windows doesn't support STDERR redirection as used in t/80_diag

2010-10-05 0.75 - H.Merijn Brand

• Fixed undefinedness of $\ in print (RT#61880)

The fix in version 0.75 is a serious fix. Version 0.76 was just released because it broke CPANTESTERS on Windows. There has been no functionality change in 0.76 at all.

I now have a graphical representation of speed differences between Text::CSV_XS and Text::CSV_PP. The first shows the difference between XS and PP. The second just shows XS method differences.

Windows doesn't support STDERR redirection as used in t/80_diag

Please forward that bug upstream

I already posted the problem on the perl5 porters mailing list. I don't know if it is Test::more or something else that is to blame.


Enjoy, Have FUN! H.Merijn

csv() is unexpectedly restrictive, and non-streaming.

It would be nice if the following would work, and processed records one at a time instead of slurping the whole file.

use Text::CSV_XS qw( csv );

csv( in => "a.csv", out => "b.csv", ... );
[download]

It does suggest how the call could be rewritten so the above works (nice), but it loads the entire file.

Honestly, I just want jq to support CSV :) (It can generate it, but not read it.)

1. The function csv () is streaming! (and well documented as such, see e.g. dumping database tables)

2. Believe me, I have had the same feeling with file-names as arguments to in and out, and I actually started coding on that, but in the end, you do not want it, as it causes too many undocumantable catch-22's.

   csv (in => csv (in => "a.csv, ...), out => "b.csv") is the best alternative and works quite well

3. My personal fav use of streaming is csv (in => $fh, out => undef, bom => 1, on_in => sub { ... process %_ ... });. If files are really large, this can be used for streaming file to file too using a second handle.


Enjoy, Have FUN! H.Merijn

Believe me, I have had the same feeling with file-names as arguments to in and out

It's the inconsistency of it.

		in
		Data Structure	Sub	File Name	File Handle
out	[Absent]	Works	Works	Works	Works
	Undefined Scalar	Works	Works	Run-time error	Run-time error
	File Name	Works	Works	Instructs the programmer	Run-time error
	File Handle	Works	Works	Run-time error	Run-time error

The instruction to the programmer is doubly weird because it could simply do what it would do if the programmer used the suggested code.

The function csv () is streaming!

You can encode as a stream.

You can decode as a stream.

But you can't transform as a stream. The inner call (csv (in => "a.csv, ...)) loads the entire file into memory.

I have just uploaded version 0.90 to CPAN, which marks a bigger step than the usual releases, as I have dropped perl-5.005 support. Perl 5.6.1 is now the minimum.

0.90 - 2012-06-15

• Drop 5.005 support (5.6.1 is now minimum)
• Introduce record_number
• Try harder to get the complete input parsed for the current record when hitting errors on parsing seekable IO (only works in 5.14.0 and up)
• Tested with perl 5.6.1 .. 5.17.0 (99 versions of perl) on Linux, HP-UX, AIX, and Windows
• SvSETMAGIC was missing for tied variables causing weird actions at a distance, e.g. in printf (Thanks TonyC for finding this)
• UTF8 flag was not always reset when using bound variables (TonyC)

0.88 - 2012-03-16

• Fix for $/ in 0.86 broke parsing fields that contain excessive $/'s

0.87 - 2012-03-08

• Extra check on utf8 output (RT#74330)
• examples/csvdiff now recognizes numerically sorted CSV files
• Document example comparing getline_hr vs bind_columns + getline

0.86 - 2012-01-22

• Introduce quote_binary attribute
• Update copyright to 2012
• Versions
• Fixed a utf8::decode on undef (found by perl-5.15.7)
• Fixed localized $/ interference with other handles (RT#74216)

0.85 - 2011-09-07

• NAME / DISTNAME in Makefile.PL

0.84 - 2011-09-07

• More cross-checks for META data

0.83 - 2011-08-07

• Fix spurious auto_diag warning (RT#69673)
• Tested with 50 versions of perl, including 5.15.1

Several small fixes, but 0.94 might fix more than you needed …

0.95 - 2013-01-13

• Introduce allow_unquoted_escape as workaround for RT#81295
• Update copyright to 2013
• Introduce print_hr () for RT#76143
• Dropped plans to support EBCDIC

0.94 - 2012-12-03

• Guard against beta releases of Encode (Xavier Guimard - RT#81499)
• Fix sv_cache init global-buffer-overflow (Reini Urban - RT#81469)
• Tested with perl compiled with clang (also to verify RT#81469)
• Fix memory leak reported in RT#81539 (Fix by Tony Cook)

0.93 - 2012-11-19

• Skip Encode related tests on too old Encode
• Force old(er) tar format (ustar) - assumes GNU tar on release box

Twitter hinted me to implement RFC7111. See syntax below. If no (serious) objections, I will make a new release soon.

1.03 - 2014-01-21

• Implement RCF7111

1.02 - 2013-09-25

• Add example for reading only a single column
• Don't store NULL in _ERROR_INPUT (RT#86217/Clone)
• Prevent double-decode in csv-check
• Add decode_utf8 attribute (default is true)

1.01 - 2013-06-16

• Cache not re-read on getline_all (RT#86155)

2009-08-07 0.66 - H.Merijn Brand

• Reordered examples in doc to show best method first
• Documentation grammatical fix (John P. Linderman, RT#46411)
• Fail if first arg to new () is not a hash ref
• Implement empty_is_undef on request of Evan Carroll
• Typo in documentation (Herwin Weststrate, RT#47613)
• error_diag () uses warn () in void context instead of STDERR
• Added auto_diag attribute (still under construction)
• FIX: reset attributes (after they have been set) with accessor

2010-02-15 0.71 - H.Merijn Brand

• Upped copyright to 2010
• Prevent double encoding: make Text::CSV_XS streams behave just like perl would (thanks ikegami for the test cases)
• Text::CSV_XS->error_diag () in void context now warns instead of doing nothing
• auto_diag also used for new () itself

2009-12-02 0.70 - H.Merijn Brand

• Add quote_space attribute
• Forbid \r and \n in sep_char, quote_char, and escape_char

2010-10-23 0.77 - H.Merijn Brand

• Internals now use warn () instead of (void)fprintf (stderr, ...)
  Now the test in t/80_diag also passes on Windows
• Better parsing for eol = \r and set as such (RT#61525)
• Workaround for AIX cpp bug (RT#62388, Jan Dubois)

A serious UTF-8 fix: output could be broken if buffer bounds cut encoded code points in half.

0.92 - 2012-11-12

• Allow bind_columns () for print ()
• Increase buffer size for print to 64k
• Fix RT#80680 - Buffer break halfway UTF8 + tests

0.91 - 2012-08-21

• Prevent test-failures for long doubles on weird architectures
• More utf-8 tests for te change of 0.90
• Update test case now 5.005 is not supported anymore
• Rip out the tell/seek introduced in 0.90

I was inevitable, but with the steady releases, I was bound to hit 1.00. Here it is …

1.00 - 2013-06-13

• Fix automatic UTF-8 in getline/parse for SV's with \0

0.99 - 2013-06-05

• Documents return value of bind_columns without arguments
• Fix automatic UTF-8 in getline/parse

0.98 - 2013-06-03

• Clarify eol documentation
• Move error_input to XS

0.97 - 2013-03-30

• Regain the speed from 0.91 (buffer back to 1k)
• Minor cleanup in XS code
• Add diag_verbose attribute

0.96 - 2013-03-26

• No need to require Test::Harness if unused (RT#82693)
• parse ("") should return one empty field, not undef
• Now that we know the record number, show it in auto_diag

2010-03-16 0.72 - H.Merijn Brand

• Introduce quote_null attribute (RT#55200)
• examples/csv-check can be used for Text::CSV_PP
• examples/csv-check more options for sep_, escape_ and quote_char
• examples/csv2xls more options for sep_, escape_ and quote_char
• examples/csv2xls added auto_diag and verbosity
• Dropped YAML spec to 1.0

I also added a test to make sure a former inconsistency between Text::CSV_XS and Text::CSV_PP is tested for.

Hi and thanks for this great module,

I was wondering if you could add an option (unless it already exists and I missed it in the documentation).

I am currently dealing with CSV files that were transmitted from a remote computer somewhere in the nature to our server. There are often lines corrupted, meaning they can have more or less values than the header. I want to skip them (because they are corrupted) but not die the program (because next lines are good). I use the bind_columns but when there are more values I get the 3006 error and it gets out of the while. Putting an error_diag right after the while loop and checking for the 3006 error, I can then say goto beginning_of_loop, but this is not the solution I prefer... And I could not check if there was enough values to match the headers...

So what I imagine is an option in the constructor letting the programer choose if the number of columns matches the header (if any) goes on without reporting, make a warning that could go invisible or fetched in error_diag or make the usual error as now (unseen if no auto_diag but dies anyway). But I let you deal with the 'how'...

XN

As of Text::CSV_XS-1.05, you can catch/ignore any error at your own risk:

          my ($c, $s);

          sub ignore3006
          {
              my ($err, $msg, $pos, $recno) = @_;
              if ($err == 3006) {
                  # ignore this error
                  ($c, $s) = (undef, undef);
                  SetDiag (0);
                  }
              # Any other error
              return;
              } # ignore3006

          $csv->callbacks (error => \&ignore3006);
          $csv->bind_columns (\$c, \$s);
          while ($csv->getline ($fh)) {
              # Error 3006 will not stop the loop
              }
[download]

Note that this API is young and new. New insights might enhance or change later on.


Enjoy, Have FUN! H.Merijn