Re^7: Pre-process csv files before using

I have not used DBI::CSV to comment on issues with "." in names. It seems logical that it will break coz of dots as . is typically not a valid name charcater.

Anyways you need to do an inplace edit of your CSV file before you pass it on to DBI::CSV. The code below demonstrates how to do an inplace edit of your file. This will create a backup of the file (mydata.csv.bak) but you can set the $^I flag to "" to pure inplace effect.

WARNING: In Place edits are dangerous. Test out the code on a sample file before you run it on the original file

{
  local $^I = ".bak";
  local *ARGV;
  @ARGV = "mydata.csv";
  while (<>) {
    s/\.//g if ($. == 1);
    print;
  }
}
[download]

Input

PDHCSV40EasternDaylightTime.240,ERWWCOMMUNITIESMemory.PagesPERsec,ERWW
+COMMUNITIESNetworkI
nterfaceEthernetAdapterModuleBytesTotalPERsec,ERWW.COMMUNITIESNetworkI
+nterface.EthernetAd
apterCurrentBandwidth
1.1,6.6,8,13
1.1,6.6,8,13
1.1,6.6,8,13
1.1,6.6,8,13
[download]

Note the header - I have threw in some dots there

After I run the program my mydata.csv becomes this -

PDHCSV40EasternDaylightTime240,ERWWCOMMUNITIESMemoryPagesPERsec,ERWWCO
+MMUNITIESNetworkInt
erfaceEthernetAdapterModuleBytesTotalPERsec,ERWWCOMMUNITIESNetworkInte
+rfaceEthernetAdapte
rCurrentBandwidth
1.1,6.6,8,13
1.1,6.6,8,13
1.1,6.6,8,13
1.1,6.6,8,13
[download]

The dots in the header are gone but the ones in the data still remain. Now your CSV file is ready for DBI.

Hope this helps!

cheers

Comment on Re^7: Pre-process csv files before using Select or Download Code

Replies are listed 'Best First'.
Re^8: Pre-process csv files before using by DrAxeman (Scribe) on Aug 07, 2005 at 02:00 UTC
I am working on test data. I can destroy this data as many times as I like! :-) As for the code, it's perfect! Thank you very much!	[reply]
Re^8: Pre-process csv files before using by DrAxeman (Scribe) on Aug 07, 2005 at 02:44 UTC
Just one quick question. How about if I wanted to keep the original file intact, and have this create a new file with the modified data (over-writting an existing file if it exists)? Say it reads, data.csv and writes a new file named <origional-file-name>-ready.csv (or something like that). This way, if there is an error, I can re-run the perl script with out having to re-copy the data. --Scratch that-- I figured out that there is a rename() function.	[reply]
Re^9: Pre-process csv files before using by sk (Curate) on Aug 07, 2005 at 03:47 UTC
You are making it complicated by doing a rename and in place edit. I would do - `open (IN, 'origdata.csv'); # Open the file for read open (OUT,'>','copy.csv'); # Open the file for write. Will overwrite +if exists while(<IN>) { # read contents of input file # <IN>, typucally used inside a while will populate variable $_ # more proecessing print OUT ($_); # Write out to new file }` [download] I think you should start reading more about how to work with files and other perl idioms. I would recommend that you start with "Learning perl book". Also there are lot of stuff in Categorized Questions and Answers section! -SK	[reply] [d/l]


Come for the quick hacks, stay for the epiphanies.
	PerlMonks