parekhdip has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I have many tab delimited MASTER files (with header) in which last 3 columns of each line contains the path of another tab delimited files (CHILD). I need to read the MASTER files line by line and get the child file. Compare a key column captured from Master file in child file get few columns and add it to master file at the end. number of COLUMNS in Master files are not known so how can we do it in perl. Any help on this is appreciated.

Replies are listed 'Best First'.
Re: Line by Line file processing
by GrandFather (Saint) on May 04, 2007 at 02:17 UTC

    Tools that may be of use in this are Text::xSV or Text::CSV and knowing about array slicing (see slices in perldata):

    use strict; use warnings; my @column = (1 .. 9); my @last3 = @column[-3 .. -1]; print "@last3\n";

    Prints:

    7 8 9

    DWIM is Perl's answer to Gödel
Re: Line by Line file processing
by siva kumar (Pilgrim) on May 04, 2007 at 05:34 UTC
    Open the MASTER file
    open(FH,"masterfile.txt") or die("Can't open the file : $!");
    Read master file contents and put it array
    @masterContents = <FH>;
    Read only the CHILD file details from master file content
    @childDetail = @masterContents[-3 .. -1 ];
    childDetail array will contain the last 3 lines of child details. Then Apply the programming logic.
Re: Line by Line file processing
by MonkE (Hermit) on May 04, 2007 at 15:51 UTC
Re: Line by Line file processing
by swampyankee (Parson) on May 05, 2007 at 13:37 UTC

    This is, in concept fairly straightforward. The details depend on information about the file format which you've not provided.

    open the master file. open a temp file. read a record, skipping header records, until a significant record or +end of file. Split the significant record (see the split function) into fields. Th +row out all but the last three fields and the key column. loop through the pointers to the child files open each child file read the child file, checking for the key value. write the appropriate data to a temp file. close the child file. repeat until all records in master file have been processed. close the master file. close the temp file open it for appending open the temp file for reading read through the temp file, copying records to the end of the master f +ile.

    The key functions to look at are open, split, and grep. Check out Perl's array operators. It probably would be best to use Tie::File, which eliminates the need for opening and closing the master file and keeping a temp file.

    In Perl, using negative array indices will count from the end; $array[-1] points to the last element in an array; $array[-2] points to the second to last, etc (symmetry would be nice, but Perl counts array elements from 0, and most computers can't distinguish 0 from -0. Alas.).

    emc

    Insisting on perfect safety is for people who don't have the balls to live in the real world.

    —Mary Shafer, NASA Dryden Flight Research Center