PERL_fresher has asked for the wisdom of the Perl Monks concerning the following question:

Hello Few Perl monks, I have very urgent requirement to perform below changes.

I have one master file a csv file delimited by commas and a directory which has several files 8 to be precise.

Master file has fields such as employee ID and name, surname of employee and other fields as well.

Similarly other csv files in directory have employee ID's and name and surnames in it. what I need to do is using master file as source search employee ID's in all files in that directory and replace name and surname in the directory.

for ex.
Master file employee ID, name, surname, other fields 1,nitin, abc,... 2, aa, bb,.. 3,cc,dd,.. 4, ee, ff
directory files
employee ID, name, surname, other fields 1,nitin1, abc1,... 2, aa1, bb1,.. 3,cc1,dd1,.. 4, ee11, ff1
I need to replace this with the data in master file for all files in directory. positions of the fields in target directory are same as I have described. I am fairly new to perl and just making my way through it. any help is greatly appreciated. sample master file
Employee Number~Category Number~Forename~Surname~Initials~NI Number~Ti +tle~Gender~Marital Status~Date of Birth~Home Address Line 1~Home Addr +ess Line 2~Home Address Line 3~Home Town/City~Home County~Home Post C +ode~Employee Contribution %~Employer Contribution %~Annual Salary~Dat +e of Joining Scheme~Date of Joining Company~Exchanged %~Scheme Joiner D51231~ORDM~Charles~Smith~C T~NP307342C~Mr~Male~S~15/10/1971~45 Board +Walk~Springer Lane~Mundesley~Norfolk~~NR28 3JK~0.00~8.00~72800.00~01/ +09/2014~04/08/2014~2.00~N F78321~SMG1~Janet~Patterson~J M~AA112233A~Ms~Female~~21/07/1975~12 She +ringham Way~Cromer~~~~NR21 9BG~0.00~31.00~320000.00~01/09/2014~25/08/ +2014~0.00~N
Sample files which is to be replaced by master file
Employee Number~Category Number~Forename~Surname~Initials~NI Number~Ti +tle~Gender~Marital Status~Date of Birth~Home Address Line 1~Home Addr +ess Line 2~Home Address Line 3~Home Town/City~Home County~Home Post C +ode~Employee Contribution %~Employer Contribution %~Annual Salary~Dat +e of Joining Scheme~Date of Joining Company~Exchanged %~Scheme Joiner D51231~ORDM~Nitin~Rane~C T~NP307342C~Mr~Male~S~15/10/1971~45 Board Wal +k~Springer Lane~Mundesley~Norfolk~~NR28 3JK~0.00~8.00~72800.00~01/09/ +2014~04/08/2014~2.00~N F78321~SMG1~AAA~BBB~J M~AA112233A~Ms~Female~~21/07/1975~12 Sheringham +Way~Cromer~~~~NR21 9BG~0.00~31.00~320000.00~01/09/2014~25/08/2014~0.0 +0~N
Here D51231 F78321 are employee nos and in master file D51231 Forename = CHARLES surname = SMITH I need to do this to sample file where D51231 Forename = Nitin surname = Rane Output should be D51231 Forename = CHARLES surname = SMITH. what I was hoping with below code is when I give directory name all files in that directory are accessible to me as tables so that I can simply use simple SQL to update source file.
my $dbh = DBI->connect('dbi:CSV:', "", "", { f_dir => "$_ENV::LogsFilesDIR/", f_ext => ".txt", eol => "\n", sep_char => "~", RaiseError => 1, }) or die $DBI::errstr; print $ceridian_raw_files; if ($ceridian_raw_files =~ m/^$_ENV::new_entrants/i) { print "inside dbi $ceridian_raw_files\n"; my $sth = $dbh->prepare("select forename from $ceridian_raw_files"); $sth->execute(); my $res = $sth->fetchall_arrayref(); use Data::Dumper; print $res;
However in my case Im not able to even initiate the object $sth. any suggestions? I tried this method with some comments from below members. apologies for any ignorance on my part as name suggests Im newbie in Perl world and relying on help from fellow perl monks :) Hello Monks Issue is now resolved. thanks for your help if anyone is intrested then below is the code
my $dbh = DBI->connect("dbi:CSV:f_dir=$_ENV::LogsFilesDIR;csv_sep_cha +r=\\~") or die $DBI::errstr; $dbh->{'csv_tables'}->{'prospects'} = { 'file' => 'cimp_new_entrant.tx +t'}; $dbh->do("UPDATE prospects SET Forename = 'Nitin'"); my $sth = $dbh->prepare("select Forename from prospects"); $sth->execute(); while (@row = $sth->fetchrow_array) { print @row; } $sth->finish( ); $dbh->disconnect( );

Replies are listed 'Best First'.
Re: CSV files compare
by Tux (Canon) on May 04, 2015 at 05:59 UTC

    Sounds like the perfect job for DBD::CSV

    Unless you show what you tried so far, I don't think people will come up with a ready-to-go suggestion.


    Enjoy, Have FUN! H.Merijn
Re: CSV files compare
by shmem (Chancellor) on May 04, 2015 at 07:37 UTC

    You open the master file and read it i line by line. See readline. Extract the employee ID either splitting the line into an array (see perldata), or via a regular expression. Insert the employee ID as key and the current line as value into a hash table (again, see perldata). Close the master file after the reading loop.

    Read the contents of the directory containing the 8 other files with opendir and readdir into an array. Loop over that array, and for each file: open the file in reading mode; open the respective output file with another file handle in writing mode; while reading each file, extract the employee ID off the current line as above, look up its value in the previous built hash table; if found, output that line , if not, the current line, to the output file handle; close both files after the inner reading loop.

    If you have problems with any of the above, come back with the respective code.

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: CSV files compare
by shmem (Chancellor) on May 04, 2015 at 11:01 UTC
    f_dir => "$_ENV::LogsFilesDIR/",

    What is in $_ENV::LogFilesDIR? If that is supposed to be an environment variable, it should be written as $ENV{LogFilesDIR} proper.

    Hint: always do

    use strict; use warnings;

    That will inform you about misformed / undeclared variables - but not in the case of $_ENV::LogFilesDIR since it is fully qualified as a $LogFilesDIR in the package/namespace _ENV - which, I suppose, is not what you intended.

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: CSV files compare
by aaron_baugher (Curate) on May 04, 2015 at 04:26 UTC

    If you want help doing it yourself, show us the code you have so far. Also show a sample of the input (a few lines of your master file and one of the other files, for instance) and the expected output from that input. Actual samples are much easier to understand than descriptions of the data. Make sure you wrap your code and sample data in <code></code> tags so it will be legible.

    Aaron B.
    Available for small or large Perl jobs and *nix system administration; see my home node.

Re: CSV files compare
by Laurent_R (Canon) on May 04, 2015 at 07:04 UTC
    Just the same comments as the other monks. Please show samples of the input files and of the desired output. And please also show us what you have tried so far, it is usually better to suggest corrections to existing code than to provide a full solution.

    Je suis Charlie.