Handling CSV files

morgon has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I am currently munching CSV files and have re-occurring tasks like

- add a line consisting of sums of column-values

- calculate things like "sum all values in column 2 execpt for the last one and then find the difference to the last one"

- find the average of the values in column 4

etc etc

and I wonder: is there a good module to do such kinds of things?

At the moment I use DBD::CVS and use SQL to calculate things but it does seem to be a bit silly...

Bascially I am looking for a module that would allow me to treat a CSV a bit like a spreadsheet, but from the command-line as the output goes straight into gnuplot.

Probably all of the above sounds like the ranting of a mad man but maybe someone understands what my problem is and can give me a hint.

Many thanks!

Comment on Handling CSV files

Replies are listed 'Best First'.
Re: Handling CSV files by kevbot (Vicar) on Oct 21, 2015 at 04:14 UTC
Take a look at Data::Table, and the Data::Table Cookbook. Here are a few nodes, where I have used Data::Table: Re: Extract a column of csv Re: Descriptive Stats from .csv file Re: Merging two data sets Re: perl: How to comapre and manipulate csv file in perl? Re: Merging Columns from multiple files into a single file	[reply]
Re: Handling CSV files by NetWallah (Canon) on Oct 21, 2015 at 02:44 UTC
How about fsql ? The description seems to match your requirements: fsql lets you perform SQL queries against one or several "flat" files of various formats. Each file will be regarded as a SQL table. By using SQL queries, you can do various calculations or manipulations that are otherwise hard/cumbersome to do with traditional text-manipulating Unix commands like cut, sort, head, tail, uniq, and so on. Particularly: data grouping, joining, or filtering with SQL expressions and functions. The best defense against logic is ignorance.	[reply]
Re^2: Handling CSV files by Tux (Canon) on Oct 21, 2015 at 06:39 UTC
That app looks fantastic, but be prepared to get half of CPAN installed, as its dependency chain is huge. (Hit the "Dependency graph" in the bottom-right corner of that link) Enjoy, Have FUN! H.Merijn	[reply]
Re^3: Handling CSV files by soonix (Chancellor) on Oct 22, 2015 at 17:13 UTC
"Service temporarily unavailable" -- slashdotted by Perlmonks? Update: JFTR this was an attempt at humour, making fun of the time coincidence. (impressive graph, BTW)	[reply]
Re: Handling CSV files by GrandFather (Saint) on Oct 21, 2015 at 22:18 UTC
Define "silly" in the context of using DBD::CVS for the task. What would "less silly" look like? Premature optimization is the root of all job security	[reply]