in reply to Merging files

use Text::CSV_XS (); my %data; parse_ssn_file( "file1.txt", \%data ); parse_salary_file( "file2.txt", \%data ); my $csv = Text::CSV_XS->new(); foreach my $cur ( sort keys %data ) { $csv->combine( $cur, @{ $data{$cur} }{qw( ssn file salary zip )} ); print $csv->string, "\n" } exit 0; sub parse_ssn_file { my( $file, $data ) = @_; local( *CUR ); open( CUR, $file ) or die "Can't open $file: $!\n"; scalar <CUR>; # discard header my $csv = Text::CSV_XS->new( ); while( <CUR> ) { $csv->parse( $_ ); my @cur = $csv->fields(); $data->{ $cur[0] }->{ssn} = $cur[1]; $data->{ $cur[0] }->{file} = $cur[2]; } return } sub parse_salary_file { my( $file, $data ) = @_; local( *CUR ); open( CUR, $file ) or die "Can't open $file: $!\n"; scalar <CUR>; # discard header my $csv = Text::CSV_XS->new( ); while( <CUR> ) { $csv->parse( $_ ); my @cur = $csv->fields(); $data->{ $cur[0] }->{salary} = $cur[1]; $data->{ $cur[0] }->{zip} = $cur[2]; } return } __END__
file1.txt Name,SSN,File "Doe, John",123456789,3447 "Flinstone, Fred",000000302,1234 file2.txt Name,Salary,ZIP "Flinstone, Fred",34500,90210 "Doe, John",50000,10003 Output: "Doe, John",123456789,3447,50000,10003 "Flinstone, Fred",000000302,1234,34500,90210

Update: D'oh, might help if I answered the whole question. :/ If you want to catch exceptions then you'd want to pick whatever file is going to be canonical and parse it first. Have all the other parse_foo_file routines first do an exists $data->{$cur} to check whether it's a valid, existing name before inserting the data. If it's not, have an open log file (open( REJ, ">$filename.rej" ) or die $!) that you can print the rejected entries into.

Replies are listed 'Best First'.
Re: Re: Merging files
by ellem (Hermit) on Oct 27, 2001 at 04:07 UTC
    Thanks! This really did clear up a lot of what I was doing wrong.
    Two questions:
    1 - Although you commented it I don't understand what
    scalar <CUR>;
    does.

    2 - Instead of using Names as the key you're using two columns? What would you suggest for files with a greater number of columns? Could I just continue on with $cur[3],[4] etc.?
    --
    lmoran@wtsgSPAM.com
    print "\x{263a}"

      As for number one, that's just throwing away the first line. It's just the normal readline diamond operator in void context (technically the scalar is superfluous, but I put it in out of habit). The return value is being discarded rather than being saved anywhere (my $toss = <CUR> would have done just the same thing with a (useless) temporary).

      As for the data structure, it's a hash of hashes. The top level hash (%data) has keys that are the primary key from your data file (the names), and the corresponding value is a hash of key/value pairs from the various data files. Add something like the following to the end of the program and you can see what it looks like:

      ... use Data::Dumper qw( Dumper ); print Dumper( \%data );

      See also perldoc perldsc.

        one last time, thank you! I adjusted the code to meet my actual files and after a few typos I made/repaired it works exactly as expected. Once it was running on my data I could see what the keys were. I am working on your update now and after a quick trip to perldoc I'm set. You rock. thanks!
        --
        lmoran@wtsgSPAM.com
        print "\x{263a}"