Re: Process and combine two CSV files into one

I try to take the KISS approach. If you don't need the SQL functionality, why not just open 2 output files - the individual .csv file and a "Master" .csv file - and print your data to both files.

# Setup the Master list
open(MASTERFILE, ">>$dir/MasterList.csv")
   or die "Can't open MasterList.csv: $!";

# Set up a loop to "Do Stuff" tm
my @src_files = <$src_dir/*.csv>;
foreach my $src_file (@src_files) {

    open (INFILE, "$src_file")
        or die "Can't open $src_file for input: $!\n";

    while (<INFILE>) {
        chomp;

    #
    # Do Stuff Here!
    #

    open (OUTFILE, ">>$dest_file")
        or die "Can't open $dest_file for output: $!\n";

    print OUTFILE "Your records go here\n";
    print MASTERFILE "Your records go here\n";

    close OUTFILE;

}

close MASTERFILE;
[download]

If you don't need the individual .csv files just leave out the appropriate open and print statements.

HTH,

Jack

Comment on Re: Process and combine two CSV files into one Download Code

Replies are listed 'Best First'.
Re^2: Process and combine two CSV files into one by DrAxeman (Scribe) on Aug 09, 2005 at 17:17 UTC
I tried something similar to this, but got unexpexcted results. (My skills with SQL is even worse than my skills with Perl.) `# Connect to the database, (the directory containing our csv file(s)) my $dbh = DBI->connect("DBI:CSV:f_dir=.;csv_eol=\n;"); # Associate our csv disk file with the table name 'results' $dbh->{'csv_tables'}->{'info'} = { 'file' => "psinfooutputfile.csv"}; $dbh->{'csv_tables'}->{'hosts'} = { 'file' => "netbiosoutputfile.csv"} +; ###### my @row = $dbh->selectall_arrayref("SELECT * FROM hosts, info WHERE ho +sts.IP = info.IP"); print Dumper @row;` [download] Produced `$VAR1 = [ [ '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10', '10.160.0.10' ],` [download] I am assuming that there is an issue with my SQL. I figured it out. I need to name each Column in my statement, not just an *. My problem is that it isn't joining data from one of the tables. # Connect to the database, (the directory containing our csv file(s)) my $dbh = DBI->connect("DBI:CSV:f_dir=.;csv_eol=\n;"); # Associate our csv disk file with the table name 'results' $dbh->{'csv_tables'}->{'info'} = { 'file' => "psinfooutputfile.csv"}; $dbh->{'csv_tables'}->{'hosts'} = { 'file' => "netbiosoutputfile.csv"} +; ###### my @row = $dbh->selectall_arrayref("SELECT IP, Domain, ServerName, Day +sUptime, OS, RAM, OSSP, InstallDate, CPUSpeed, CPUCount, CPUType FROM + hosts, info WHERE hosts.IP = info.IP"); print Dumper @row; [download] Output; `$VAR1 = [ [ '10.160.0.10', '0', '', '19', 'Microsoft_Windows_2000_Server_Domain_Controller', '1024', '4', '12_2_2003', '995', '1', 'Intel_Pentium_III' ], ]` [download] UPDATE: I modified the top line of the file who's data wasn't getting joined and found that the data included there was the ONLY data being used. I changed it to an INNER JOIN and it looks like it's working.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: Process and combine two CSV files into one
by DrAxeman (Scribe) on Aug 09, 2005 at 17:17 UTC

# Connect to the database, (the directory containing our csv file(s))
my $dbh = DBI->connect("DBI:CSV:f_dir=.;csv_eol=\n;");

# Associate our csv disk file with the table name 'results'
$dbh->{'csv_tables'}->{'info'} = { 'file' => "psinfooutputfile.csv"};
$dbh->{'csv_tables'}->{'hosts'} = { 'file' => "netbiosoutputfile.csv"}
+;

######
my @row = $dbh->selectall_arrayref("SELECT * FROM hosts, info WHERE ho
+sts.IP = info.IP");
print Dumper @row;
[download]

$VAR1 = [
          [
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10',
            '10.160.0.10'
          ],
[download]

# Connect to the database, (the directory containing our csv file(s))
my $dbh = DBI->connect("DBI:CSV:f_dir=.;csv_eol=\n;");

# Associate our csv disk file with the table name 'results'
$dbh->{'csv_tables'}->{'info'} = { 'file' => "psinfooutputfile.csv"};
$dbh->{'csv_tables'}->{'hosts'} = { 'file' => "netbiosoutputfile.csv"}
+;

######
my @row = $dbh->selectall_arrayref("SELECT IP, Domain, ServerName, Day
+sUptime, OS, RAM, OSSP, InstallDate, CPUSpeed, CPUCount, CPUType FROM
+ hosts, info WHERE hosts.IP = info.IP");
print Dumper @row;
[download]

$VAR1 = [
          [
            '10.160.0.10',
            '0',
            '',
            '19',
            'Microsoft_Windows_2000_Server_Domain_Controller',
            '1024',
            '4',
            '12_2_2003',
            '995',
            '1',
            'Intel_Pentium_III'
          ],
]
[download]

[reply]
[d/l]
[select]