Re^2: How to go about this?

Ok, I've come a lot further since I last followed up on the replies I've gotten. Below is the script that I have. Here's the low down. I have a master file and it has some lines in that I'm trying to match to another file; the other file has a field in it that I would like to place in the master file. Both files have a key that I'm using to compare the rows and that's actually an IP Address.
My Problem is that there are some records that are in the master file that are not in the other file and when I check my output those records don't show. So that's one thing I'm looking for and the other works the other way around in that there are some records in the other file that are not in the master file. Pretty much I want every line from both files in the output file.
If there is anyone that can look at the code below and modify it to where it will do just that?

#!perl
#perl "copy of findandreplaceemail2.pl" -d . -i "Reporting Devices_01-
+03-2009-09-00-00.csv" -s "Master Sched.csv" -o "Master Sched(new1).cs
+v"

use Getopt::Long;
use Text::CSV;

&GetOptions('d=s'   => \$baseDir
           ,'i=s'   => \$inFlNm
           ,'s=s'   => \$subFlNm
           ,'o=s'   => \$outFlNm
           ,'h'     => \$hlp
           ,'hlp'   => \$hlp
           ,'help'  => \$hlp);
       
$emailList = "test.csv";
$csv = Text::CSV->new();

# Save substitution data in memory indexed by manager email address
open IN1, "<"."./$baseDir/$subFlNm" or die "Can't open input file >$su
+bFlNm<\n";

open OUT, ">"."$baseDir/$outFlNm" or die "Can't open output file >$out
+FlNm<\n";

while(<IN1>){
    chomp;
    $_ =~ s/\,\,$//;
    $csv->parse($_);
    @subIn = $csv->fields();
    $subRcd{$subIn[2]} = $_;
    #print "@subIn\n";
}
close IN1;

#There's got to be a way to take all of the Master file and use it in 
+the while loop

open IN1, "<"."./$baseDir/$subFlNm" or die "Can't open input file >$su
+bFlNm<\n";

while ($in1 = <IN1>){
    chomp($in1);
    @mainFlds = split(/,/, $in1);
    $mainFlds[2] =~ s/^\s+//; $mainFlds[2] =~ s/\s+$//;
    if($in1 =~ /\d+\.\d+\.\d+\.\d+/){
        open IN2, "<"."$baseDir/$inFlNm" or die "Can't open input file
+ >$inFlNm<\n";
        while ($in2 = <IN2>){
            chomp($in2);
            @subFlds = split(/,/, $in2);
            if($in2 =~ /\d+\.\d+\.\d+\.\d+/){
                $subFlds[1] =~ s/^\s+//; $subFlds[1] =~ s/\s+$//;
                if("$mainFlds[2]" eq "$subFlds[1]" && "$mainFlds[1]" e
+q "$subFlds[0]"){
                    if($in1 =~ /,,$/ || $mainFlds[7] ne '' || $mainFld
+s[8] ne ''){
                        print OUT "$in1,$subFlds[5]\n";
                    }
                    elsif ($mainFlds[6] ne ''){
                        print OUT "$in1,,$subFlds[5]\n";
                    }
                    else {
                        print OUT "$in1,,,$subFlds[5]\n";
                    }
                }
            }
        }
        close IN2;
    }
    elsif(($mainFlds[2] eq '' || $mainFlds[3] eq '') && $mainFlds[5] n
+e '') {
        open IN2, "<"."$baseDir/$inFlNm" or die "Can't open input file
+ >$inFlNm<\n";
        while ($in2 = <IN2>){
            chomp($in2);
            @subFlds = split(/,/, $in2);
            if($subFlds[1] eq ''){
                #print ">@subFlds\n";
                $subFlds[1] =~ s/^\s+//; $subFlds[1] =~ s/\s+$//;
                if("$mainFlds[3]" eq "$subFlds[2]" && "$mainFlds[4]" e
+q "$subFlds[3]"  && "$subFlds[4]" ne ''){
                    if($in1 =~ /,,$/ || $mainFlds[7] ne '' || $mainFld
+s[8] ne ''){
                        print OUT "$in1,$subFlds[5]\n";
                    }
                    elsif ($mainFlds[6] ne ''){
                        print OUT "$in1,,$subFlds[5]\n";
                    }
                    else {
                        print OUT "$in1,,,$subFlds[5]\n";
                    }
                }
            }
        }
        close IN2;
    }
    else{
        if($in1 =~ /Date/){
            @date = split(/_/, $inFlNm);
            $date[1] =~ s/\-\d\d\-\d\d\-\d\d\.csv//;
            print OUT "$in1,$date[1]\n";
        }
        else{
            print OUT "$in1\n";
        }
    }
    next;
}
[download]

Comment on Re^2: How to go about this? Download Code

Replies are listed 'Best First'.
Re^3: How to go about this? by Plankton (Vicar) on Feb 12, 2009 at 22:48 UTC
I am big fan of SQLite. If I were you I'd load each of your CVS files into SQLite relation database and deal with the data via SQL.	[reply]
Re^3: How to go about this? by graff (Chancellor) on Feb 13, 2009 at 05:47 UTC
You seem to be loading your first csv file into a hash called `%subRcd`, but then you are never using that hash anywhere else in the code (it's "write-only"). Maybe you should read your second file into that same hash (assuming that the contents of both files are structured the same way, or at least that it's easy to tell them apart in terms of their contents). When the second file has a key field that matches something in the first file, you'll need to update the hash value for that key, in order to combine the info from the two files into the one record. Apart from that, keys that occur only in the "master" will remain unmodified in the hash, and keys that occur only in the second file will be loaded and kept as-is in the hash. Then just print out all the hash elements to get the complete union of contents from the two files. (But I really like Plankton's idea of using SQLite -- definitely worthwhile. Also, you really should start with `use strict;` and `use warnings;` -- it will make you a better programmer, and you'll grow to appreciate it.)	[reply] [d/l] [select]