in reply to Batch processing of files

I can not see any obvious problem in your code, but you should check the status of your open and rename calls, just in case they are failing:
open(TMP,">$tmp") or die "error opening file $tmp: $!";
See also Basic debugging checklist

Replies are listed 'Best First'.
Re^2: Batch processing of files
by ikegami (Patriarch) on Feb 11, 2010 at 18:44 UTC
    Cleaned up code with error checking:
    use strict; use warnings; use Fcntl qw( SEEK_SET ); my @files = <files*.txt>; # Should really use File::Temp my $fn_out = "tmp.txt"; my $error = 0; for my $fn_in (@files) { open(my $fh_out, '>', $fn_out) or die("Can't create output file \"$fn_out\": $!\n"); open(my $fh_in, '<', $fn_in) or do { warn("Can't open input file \"$fn_in\": $!\n"); $error = 1; next; }; my $sum = 0; my $cnt = 0; while (<$fh_in>) { chomp; $sum += ( split /\|/ )[2]; ++$cnt; } my $avg = $sum / $cti; seek($fh_in, 0, SEEK_SET) or do { warn("Can't seek in input file \"$fn_in\": $!\n"); $error = 1; next; }; while (<$fh_in>) { chomp; my ($hd, $md, $tl) = split(/\|/, $_); my $tlf = sprintf("%.4f", $tl/$avg); print $fh_out "$hd|$md|$tlf\n"; } close($fh_in); close($fh_out) or do { warn("Can't save to output file \"$fn_out\": $!\n"); $error = 1; next; }; rename($fn_out, $fn_in) or do { warn("Can't rename \"$fn_out\" to \"$fn_in\": $!\n"); $error = 1; next; }; } exit($error);

      Thanks everyone.

      I think I found out what the problem was. Looks like I had to close all the sub-files, before populating the @files array.

      close(filehandle); #close newly generated files before array call. my @files = <substituent_R*.txt>;
      Austin-

        Hi all,

        I just tested the code from ikegami in my script, and the last file was still not processed. I guess the key is, when I generate the sub-files separately, and then use this code, all the sub-files are processed. However, when include this code in the entire script, only the first 3 out of 4 files are processed. Is there a reason for this behaviour?

        I could include the entire script if that would help

        Thanks, Austin-
        my @files = <substituent_R*.txt>; my $fn_out = "tmp.txt"; my $error = 0; for my $fn_in (@files) { open(my $fh_out, '>', $fn_out) or die("Can't create output fil +e \"$fn_out\": $!\n"); open(my $fh_in, '<', $fn_in) or do { warn("Can't open input file \"$fn_in\": $!\n"); $error = 1; next; }; my $sum = 0; my $cnt = 0; while (<$fh_in>) { chomp; $sum += ( split /\|/ )[2]; ++$cnt; } my $avg = $sum / $cnt; seek($fh_in, 0, SEEK_SET) or do { warn("Can't seek in input file \"$fn_in\": $!\n"); $error = 1; next; }; while (<$fh_in>) { chomp; my ($hd, $md, $tl) = split(/\|/, $_); my $tlf = sprintf("%.4f", $tl/$avg); print $fh_out "$hd|$md|$tlf\n"; } close($fh_in); close($fh_out) or do { warn("Can't save to output file \"$fn_out\": $!\n"); $error = 1; next; }; rename($fn_out, $fn_in) or do { warn("Can't rename \"$fn_out\" to \"$fn_in\": $!\n"); $error = 1; next; }; } exit($error);