in reply to Opine on multiple open filehandles vs. multiple loops through data

Since the log files are sorted, there's no reason to have more than two handles open.

while (my $logfile = glob('access-????.log')) { open(my $logfile_fh, '<', $logfile) or die("Unable to open yearly logfile $logfile: $!\n"); my ($year) = $logfile =~ /^access-(....)\.log$/; my $output_fh; my $month = -1; while (<$logfile_fh>) { # Extract month of log entry. my ($new_month) = /$year-(..)/; if ($new_month != $month) { $month = $new_month; my $output_file = sprintf('access-%04d-%02d.log', $year, $mon +th); open($output_fh, '>', $output_file) or die("Unable to create monthly logfile $output_file: $!\ +n"); } print $output_fh $_; } }
  • Comment on Re: Opine on multiple open filehandles vs. multiple loops through data
  • Download Code

Replies are listed 'Best First'.
Re^2: Opine on multiple open filehandles vs. multiple loops through data
by aboyd (Sexton) on Jan 20, 2006 at 21:20 UTC
    Sadly, the files aren't perfectly sorted. At the month changeover, there are dozens of lines out of order. I assume that the server wrote the lines in batches, and didn't bother with FIFO.

    However, the bulk of each month is ordered properly. I think trying it your way would be fine -- even if it has to close/open/reopen files 20 times during the change of each month, that's still probably A-OK. Overall, that's just maybe 750 open/closes. Oh. Maybe that is a bit much. Hmm.

    Well, it's something to test out, to see how speedy it is. Thank you. :)