Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi

I have a CSV file which contains a long list of wines and spirits. I am using Text::CSV_XS to load the data into a database. However my problems arise when the wines use foreign characters which fall outside the allowed range.

Taken from the docs

"Allowable characters within a CSV field include 0x09 (tab) and the inclusive range of 0x20 (space) through 0x7E (tilde). In binary mode all characters are accepted, at least in quoted fields"

I assume this is the problem anyway, anyone got any ideas how to get around this? Is there a hack I can make which will allow me to use this module? SHould I replace all dodgy foreign accents etc, before processing...? At least the file would work then? Or is there another CSV module that anyone recommends? I've used quite a few now, and keep changing for various different reasons... I really liked this one until now.

Thanks guys
Tom

Replies are listed 'Best First'.
Re: Accent Characters and Text::CSV_XS
by borisz (Canon) on Jan 21, 2005 at 12:44 UTC
    Have you tried to open Text::CSV_XS in binary mode? 
    Text::CSV_XS->new(binary => 1,...);

    [download]
    Another good module is Text::xSV.
    Boris
[reply]
[d/l]
      Yes, borisz is correct, if you start Text::CSV_XS in binary mode, all of the characters will be accepted. Minor caveat - the new() method takes a hashref, not a hash, so the correct way is: 
       my $csv = Text::CSV_XS->new( {binary=>1} );

      [download]

      The next release of Text::CSV_XS will provide an option for defaulting for this behaviour rather than having to declare it on a script-by-script basis.

[reply]
[d/l]

        thanks for the help, I will try that out.

        cheers,
        Tom

[reply]
Re: Accent Characters and Text::CSV_XS
by existem (Sexton) on Jan 21, 2005 at 12:22 UTC
    sorry that last post was by me, I forgot to login (again! doh)
[reply]

Back to Seekers of Perl Wisdom