Re: CSV Manipulation

Or if you want to go a bit more "high level", look into DBD::CSV.

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Comment on Re: CSV Manipulation

Replies are listed 'Best First'.
Re^2: CSV Manipulation by Tanktalus (Canon) on Nov 16, 2011 at 20:09 UTC
This. If I have a choice, and I usually do, I use DBD::CSV. It makes me think about the data as data rather than a string, and has the added advantage of making transitioning to SQLite or a full-fledged RDBMS (DB2, Oracle, etc.) easier.	[reply]
Re^3: CSV Manipulation by Tux (Canon) on Nov 17, 2011 at 09:47 UTC
One note here. The OP did not mention the size of the CSV data file. DBD::CSV uses Text::CSV_XS under the hood, but it will have to read the complete file into memory to be able to do any database-like operations. With a file of 2Gb, that might result in say 20Gb of memory use (perl overhead). When files are that big - again, I don't know how large the file of the OP is - switching to basic streamed IO processing is usually a lot easier. I fully agree though that DBD::CSV is the best step towards RDBMS's where those memory limits are not applicable (for the end-user script). YMMV update: I just did a quick test with the OP data extended to a 1Mb CSV file. Reading that into memory using getline_all () resulted in a 10Mb data structure (reported by Devel::Size::total_size ()). Enjoy, Have FUN! H.Merijn	[reply]