Re: ?? Blazes under Linux; crawls under XP & Win2K...

Don't open and close the output files for every line. Open them at the start of the run, and close them at the end of your program (optional). Also, error checking is always a good idea:

my $reportfile = '/home/user/data.txt';
open my $reports, "<", $reportfile
  or die "Could not read '$reportfile': $!";

my @knowntypes = qw(1 2 3 4 5);
my %filehandle;
for my $type (@knowntypes) {
  open $filehandle{$type}, ">>", "form_${type}_record.txt"
    or die "Couldn't create 'form_$type_record.txt': $!";
};
for (<$reports>) {
  /^(\d+)\|/
    or die "Malformed input line: $_";
  my $type = $1;
  if (not exists $filehandle{$type}) {
    die "Unknown record type $type in line $_";
  };
  print $filehandle{$type} $_;
};
[download]

If you hit the file descriptor limit of your OSes (255 on Win32, some other number on Linux), there is a module for that, FileCache. The Perl Cookbook has a recipe detailing its use (7.17).

Update: ChemBoy spotted a typo. "form_${type}_record.txt" instead of "form_$type_record.txt". That's why we use strict;.

Comment on Re: ?? Blazes under Linux; crawls under XP & Win2K... Select or Download Code

Replies are listed 'Best First'.
Re^2: ?? Blazes under Linux; crawls under XP & Win2K... by tweetiepooh (Hermit) on Jan 09, 2006 at 13:44 UTC
You could extend not opening/closing so that the file is only opened if not already open. That way you don't have to know types before the run. `... while (<more data>) { if (file not already open) { open file or handle error } write data to relevent output; } close outputs;` [download]	[reply] [d/l]
Re^2: ?? Blazes under Linux; crawls under XP & Win2K... by Anonymous Monk on Jan 09, 2006 at 19:35 UTC
print can also be an expensive process. Instead of doing a print in the for loop save the resulting line to a $variable and after the loop is completed print the string to the file. `my $output; while (<report>) { $output .= $line; } print FILE $ouput;` [download]	[reply] [d/l]
Re^2: ?? Blazes under Linux; crawls under XP & Win2K... by WordWeaver (Acolyte) on Jan 13, 2006 at 10:36 UTC
Opening and closing the output files only once per run solves the problem! There's much for me to learn from all the code examples offered in this thread. Even in my halting, pidgin Perl code, though, opening and closing output files only once per run made a world of difference. I'm now getting run times under Win32 comparable to what I was getting under Linux. Not directly a Perl question, but I'm curious why Linux handled my badly written code so much more effectively than did XP.	[reply]