in reply to Re: Reversing A File
in thread Reversing A File

Your reverse puts all 1 million lines in memory.

That's on top of the 2 million indexes you place in memory when only 1 million are needed.

Fix:

use Tie::File; tie(my @data, 'Tie::File', $fname_in) or die("Unable to open \"$fname_in\": $!\n"); open(my $fh_out, '>', $fname_out) or die("Unable to create \"$fname_out\": $!\n"); for (my $i=@data; $i--; ) { print $fh_out $data[$i]; } untie @data;

Although I recommend File::ReadBackwards.

use File::ReadBackwards qw( ); my $fh_in = File::ReadBackwards->new($fname_in) or die("Unable to open \"$fname_in\": $!\n"); open(my $fh_out, '>', $fname_out) or die("Unable to create \"$fname_out\": $!\n"); while (defined(my $line = $fh_in->readline())) { print $fh_out $line; }

Replies are listed 'Best First'.
Re^3: Reversing A File
by camelcom (Sexton) on Dec 12, 2007 at 13:40 UTC
    Unfortunately, this firm doesn't have File::ReadBackwards and it's a real pain getting approval for non-core modules!

    I tried the Tie::File idea (thanks for that), but killed the job after 10mins of max'd out cpu (it was about half way through) - the file is only 180MB

    There must be a faster way??

      Ten minutes to get half way through processing a 180MB file? What spec of machine are you running this on? Have you tried something like tail -r filename.log >> backwards.log (if your OS supports it) and compared the time that it takes to finish?

      Martin
      File::ReadBackwards is pure perl, all you have to do is copy the source to a file...what's the difference between that and writing your own module that does the same thing (besides the amount of work)?