ioana007 has asked for the wisdom of the Perl Monks concerning the following question:

I have N files, each of them of the form: 
line_number var1 var2 .... varN

[download]
I am reading all the files from an ARGV list 
for my $file (@files) {
while (<IN>)
$line_number= $list[0];
push @($var1{$i}), $list[1];
etc...
}
close IN;
}

[download]
I want to create for each variable an output file that would contain something like: 
for my $j (0...Nlines-1){
open(OUT, ">", "./var1.dat")
print OUT "$line_number \t $var1{file1}[$i] \t $var1{file2}[$i] \t ...
+ $var1{fileN}[$i]\n"
}
close OUT;

[download]
What is the way to write in the same line of the output file
in different columns the information for the same
variable taken from different input files?

Is there a way to open a $NEWLINE variable and keep appending
to it values from each of the input file and, after reading
from all the input files, closing the line
and then writing the line in the output file?

Thank you,
Ioana

Replies are listed 'Best First'.
Re: append to a line arrays read from N different input files
by dogz007 (Scribe) on Aug 16, 2007 at 23:51 UTC
    I created a quick solution to your problem that loads all of the data from every file into a 3D array. It then outputs all of the data into separate files for each variable. See below.

    use strict;
my @file = qw(file1.txt file2.txt file3.txt);

# read all files into one big 3D matrix
my $data = [];
for my $i (0 .. $#file) {
  open DATA, '<', $file[$i] or die;
  while (<DATA>) {
    chomp;
    my @line = split /\s+/, $_;
    my $linenum = shift @line;
    $data->[$_][$linenum][$i] = shift @line for (0..$#line);
  }  
}

# output to files for each variable
for my $n (0 .. $#$data) {
  open FILE, '>', "var$n.txt" or die;
  select FILE;
  for my $linenum (0 .. $#{$data->[$n]}) {
    print join "\t", $linenum, @{$data->[$n][$linenum]}, "\n";
  }
  close FILE;
}

    [download]

    I fed the above code with file1.txt, file2.txt, and file3.txt all identical to the file below:

    0 0 1 2 3 4 5
1 0 1 2 3 4 5
2 0 1 2 3 4 5

    [download]

    Which then produced the following files:

    Read more... (2 kB)

    I'm sure that there might be a more memory efficient method, perhaps using PDL. However, if your files are not extremely large, then you should be fine.

[reply]
[d/l]
[select]
Re: append to a line arrays read from N different input files
by graff (Chancellor) on Aug 17, 2007 at 02:10 UTC
    I'm sorry, but I don't understand what "$i" is supposed to be. Your code snippets never show any value being assigned to it, but it seems to be an important value for indexing things.

    I'm also unclear on the nature of the data, and how this affects your goal. Do all input files have the same number of columns? The same number of lines? (Could an input file have a wrong number of columns or lines, and what should happen in that case?) How is the size of the output files (columns and lines) related to the sizes of the input files?

    If the inputs are really consistent, something like this could work (maybe): 

    #!/usr/bin/perl

use strict;

my @files = qw/foo bar/;  # get file names from somewhere
my @matrix;

for my $f ( 0 .. $#files ) {
    open( IN, "<", $files[$f] ) or die "$files[$f]: $!\n";
    while (<IN>) {
        my ( $line, @vals ) = split;
        for my $v ( 0..$#vals ) {
            $matrix[$v][$line][$f] = $vals[$v];
        }
    }
}
for my $v ( 0..$#matrix ) {
    open( OUT, ">", "var.$v" ) or die "var.$v: $!\n";
    for my $line ( @{$matrix[$v]} ) {
        print OUT join( "\t", $line, @{$matrix[$v][$line]} ), "\n";
    }
    close OUT;
}

    [download]
    (not tested)

    (update: I see now that this differs only in style from the code suggested by dogz007 -- it's the same basic approach. Sorry about being redundant.)

[reply]
[d/l]

Back to Seekers of Perl Wisdom