reset filehandle position on a buffered pipe in

johnvandam has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I need to process BIG files which for disk-size purposes I have to keep zipped. So I'm piping my data into my perl script using gunzip... For my script to work I need to sometimes peep one line ahead but then reset the file handle position to current position - 1 line. When using normal files my script works but somehow the seek function fails when I open a pipe instead of a file... I have made a test script which replicates the problem..

#!/usr/bin/perl

use warnings;
use strict;

# Test linereading

open (TEST, 'test.txt') or die "Unable to open test.txt file: $!\n";
#open (TEST, "gunzip -c test.txt.gz |") or die "Unable to open pipe: $
+!\n";
while (<TEST>) {
    print "$_";
    seek(TEST, -length($_), 1); # place the same line back onto the fi
+le handle
}
[download]

The test.txt looks like this:

This is the first line
This is the second line
This is the third line

If I open the 'file' test.txt as normal this program should give me an eternal loop printing only "This is the first line";

If I now instead of open 'test.txt' open a pipe using gunzip to give me the data in test.txt.gz the script will print each line and exit after the last. Which means that the 'go back one line didn't work.

Can anybody tell me how to deal with this and go back 1 line in a file handle for a pipe? Thank you all in advance!

Regards,
John van Dam

Comment on reset filehandle position on a buffered pipe in Download Code

Replies are listed 'Best First'.
Re: reset filehandle position on a buffered pipe in by jethro (Monsignor) on Nov 03, 2008 at 15:09 UTC
AFAIK seek doesn't work in pipes. Because pipes are not random access but simple FIFOs. The best option for you would be to implement a buffer yourself. Simple as that: `my @buffer=(); sub read_a_line { if (@buffer) { return shift @buffer; } else { my $x= <PIPE>; return $x; } } sub push_back_line { push @buffer, @_; }` [download]	[reply] [d/l]
Re^2: reset filehandle position on a buffered pipe in by johnvandam (Acolyte) on Nov 03, 2008 at 16:00 UTC
Thanks a lot! The custom buffer worked like a charm!	[reply]
Re: reset filehandle position on a buffered pipe in by Fletch (Bishop) on Nov 03, 2008 at 15:12 UTC
Pipes (and sockets and FIFOs) aren't seekable on most *NIX flavours I believe, so you're going to have to do your own buffering of some form. The `less` pager for instance spools piped input to a temporary file and then seeks on that. You could do something similar, or if you really only need one line just write your own one-line-buffering wrapper and use that to read rather than using your pipe filehandle directly. The cake is a lie. The cake is a lie. The cake is a lie.	[reply]