in reply to CSV headers. Feedback wanted

Quite an interesting idea. I think it makes sense.

Just a short slightly off-topic comment. I am almost never using Text::CSV_XS for one single reason: in 95% of the cases where I would have a use for it, my data files are just too large to fit into memory. And even when they might fit, I can't take the risks of having a program failure because one day the input file is larger than expected. What I would really wish is an iterator version of Text::CSV_XS.

Our CSV files are simple, I can work around my large file problem by just reading files line by line and using split, but I think it is a pity that Text::CSV_XS does not offer an iterator mode.

Sorry for being off-topic, but I thought it was worth mentioning this problem with Text::CSV_XS. (And, BTW, I do not feel at this point like starting to write an XS module as an alternative.)

Replies are listed 'Best First'.
Re^2: CSV headers. Feedback wanted
by Tux (Canon) on Feb 13, 2016 at 20:58 UTC

    The default low-level mode is iterative!

    The new csv function by default is pulling all data into memory, but by using a filter you can prevent that. The documentation shows examples.

    Iterative (straight from the docs):

    use Text::CSV_XS;
my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
open my $fh, "<", "file.csv" or die "file.csv: $!";
while (my $row = $csv->getline ($fh)) {
    # do something with @$row
    }
close $fh or die "file.csv: $!";

    [download]

    Enjoy, Have FUN! H.Merijn
[reply]
[d/l]
[select]
      Really? Strange, I can swear that I tried it for at least a full day and it failed miserably each time the file was really large. I guess I must have used it wrongly then, possibly I used an example that was not adequate for my needs, but it's too long ago for me to remember the exact details.

      Well, thank you and thans to AnonyMonk for the information, I'll look at it again. And... sorry for the disruption with apparently inaccurate information.

[reply]
[d/l]
Re^2: CSV headers. Feedback wanted
by Anonymous Monk on Feb 13, 2016 at 20:53 UTC
    what? The module is all about iterators, only sub csv slurps
[reply]
Re^2: CSV headers. Feedback wanted
by Anonymous Monk on Feb 13, 2016 at 20:52 UTC
    what? The module is all about iterators, only sub csv slurps
[reply]

In Section Meditations