Using File::ReadBackwards or equivalent on pre-existing file handle

HYanWong has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Using File::ReadBackwards or equivalent on pre-existing file handle by choroba (Cardinal) on Jun 25, 2014 at 21:01 UTC
You can use tell to get the line beginnings and store them in an array. Once you reach the end of the input, use seek to retrieve the lines: `#!/usr/bin/perl use warnings; use strict; my @positions; open my $OUT, '+>', '2.out' or die $!; for my $i (1 .. 20) { unshift @positions, tell $OUT; print {$OUT} "line $i\n"; } open my $BACK, '>', '2.back' or die $!; for my $pos (@positions) { seek $OUT, $pos, 0; my $line = <$OUT>; print {$BACK} $line; }` [download] Update: Used the `+>` mode to keep the filehandle open. لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ	[reply] [d/l] [select]
Re^2: Using File::ReadBackwards or equivalent on pre-existing file handle by HYanWong (Acolyte) on Jun 25, 2014 at 23:00 UTC
Somewhat sneaky, thanks. I was trying to avoid intermediate storage, as in the worst case I need to store 2.5 million positions (lines in the file), but I guess that's doable on modern machines.	[reply]
Re: Using File::ReadBackwards or equivalent on pre-existing file handle by Laurent_R (Canon) on Jun 25, 2014 at 22:42 UTC
If it is only many megabytes, just store the lines into an array and use the `reverse` function on the array to print them backward. Might not work if it is many gigabytes, but that's not what you asked.	[reply] [d/l]
Re^2: Using File::ReadBackwards or equivalent on pre-existing file handle by HYanWong (Acolyte) on Jun 25, 2014 at 23:25 UTC
p.s. am I better unshifting the lines then doing a `print(FH) foreach (@arr);` or pushing the lines then doing a `print(FH) foreach (reverse @arr);`? I suppose I'm asking, is pushing faster than unshifting (I assume yes), and does perl make a new array copy when it does a `foreach (reverse ...)`, or is it clever enough just to iterate backwards over the array. I presume it's clever...	[reply] [d/l] [select]
Re^3: Using File::ReadBackwards or equivalent on pre-existing file handle by Anonymous Monk on Jun 26, 2014 at 00:57 UTC
To sure, profile & benchmark the variations. `print ... for reverse @list` will generate a new, complete list I think (currently cannot find the right B magic to produce the optree display supporting the claim).	[reply] [d/l]
Re^3: Using File::ReadBackwards or equivalent on pre-existing file handle by Laurent_R (Canon) on Jun 26, 2014 at 06:19 UTC
Difficult to say without testing. I think that slurping the file from the FH directly into an array will be slightly faster than `push`ing or `unshift`ing it line by line. Then `pop`ing the lines to be printed might be faster than using `reverse`, but only a benchmark can really say. You might even get diffrent results with differents versions of Perl. Overall, the difference is likely to be small or almost unsignificant.	[reply] [d/l] [select]
Re^2: Using File::ReadBackwards or equivalent on pre-existing file handle by HYanWong (Acolyte) on Jun 25, 2014 at 23:02 UTC
It's only a 70Mb text file, so saving in memory is probably OK actually. Cheers	[reply]
Re: Using File::ReadBackwards or equivalent on pre-existing file handle by Anonymous Monk on Jun 25, 2014 at 21:06 UTC
I don't see why that wouldn't work. File::ReadBackwards seems to just open a file, there should be no reasons why Perl can't have multiple filehandles for one file. Did you try?	[reply]
Re^2: Using File::ReadBackwards or equivalent on pre-existing file handle by HYanWong (Acolyte) on Jun 25, 2014 at 22:47 UTC
Thanks. Good point, but I have tried it, and it doesn't seem to work. I was also hoping to reuse the same file handle, since I already have it open, which saves having to check twice for success on opening, and (I presume) closing two filehandles. Seems better to avoid the possibility of someone deleting the file between the first and the second open calls, by simply having one "open".	[reply]