truncate a file from top

SiLiK has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: truncate a file from top by Happy-the-monk (Canon) on Nov 02, 2004 at 13:13 UTC
without ... tieing the file to an array Out of curiosity, the FAQ says that's the way to go, why don't you want that? The answer is to use a `while` loop, throwing away the first bytes/lines as you go. Cheerio, Sören	[reply]
Re: truncate a file from top by zentara (Cardinal) on Nov 02, 2004 at 14:11 UTC
There are alot of ways to do this. Here is one way. This example just grabs a chunk off of the end.(truncating is left to you). `#!/usr/bin/perl -w # linux only, # usage tailz filename use strict; my $filename = shift or die "Usage: $0 file \n"; my $byte; # Open the file in read mode open FILE, "<$filename" or die "Couldn't open $filename: $!"; # Rewind from the end of the file until count eol's seek FILE,0, 2; #go to EOF seek FILE,-2048,2; #get last 2k bytes $/=undef; my $tail = <FILE>; print "$tail\n"; exit;` [download] or here is a variation for lines #!/usr/bin/perl -w # example for files with max line lengths < 400, but it's adjustable # usage tailz filename numberoflines use strict; die "Usage: $0 file numlines\n" unless @ARGV == 2; my ($filename, $numlines) = @ARGV; my $chunk = 400 * $numlines; #assume a <= 400 char line(generous) # Open the file in read mode open FILE, "<$filename" or die "Couldn't open $filename: $!"; my $filesize = -s FILE; if($chunk >= $filesize){$chunk = $filesize} seek FILE,-$chunk,2; #get last chunk of bytes my @tail = <FILE>; if($numlines >= $#tail +1){$numlines = $#tail +1} splice @tail, 0, @tail - $numlines; print "@tail\n"; exit; [download] I'm not really a human, but I play one on earth. flash japh	[reply] [d/l] [select]
Re^2: truncate a file from top by Anonymous Monk on Nov 03, 2004 at 12:22 UTC
`seek FILE,0, 2; #go to EOF seek FILE,-2048,2; #get last 2k bytes` [download] Not much point in seeking twice is there? The latter seek, seeks to 2048 bytes from the end - no reason to first seek to the end. And for readability (and portability) reasons, I would use `seek FILE, -2048, SEEK_END;`, importing the constant from Fcntl.	[reply] [d/l] [select]
Re: truncate a file from top by Anonymous Monk on Nov 02, 2004 at 13:21 UTC
You no need to tie the file to an array, or load the file to an array to limit a file to N lines or bytes. You do however have to read in the file one way or another (unless you modify the filesystem directly), although not all at once. Maybe the simplest way is to call `tail`: `tail -n N file > file.$$; mv file.$$ file # Keep last N lines tail -c N file > file.$$; mv file.$$ file # Keep last N bytes` [download]	[reply] [d/l]
Re: truncate a file from top by periapt (Hermit) on Nov 02, 2004 at 13:15 UTC
I assume that you want to preserve the older entries and just overwrite existing ones. The relevent question in this case is whether each line or log block is of fixed size. For example, if you structure your log messages to be, say, 384 bytes or less (any size will do), then open your log in r/w mode. If the file is of the correct size, position the pointer at the beginning of the file and begin writing your standard blocks. If the file is not yet of the correct size, position the file pointer at the end of the file and keep adding. The only thing you need is a MAXSZ, CURRSZ and BLOCKSZ variable to track location. I'm sure there are other ways but this was the first one that came to mind. PJ use strict; use warnings; use diagnostics;	[reply]
Re: truncate a file from top by Roger (Parson) on Nov 02, 2004 at 15:06 UTC
I think you should take a look at one of the log file rotation modules on CPAN, such as Logfile::Rotate. No need to redo the work since it has already been done.	[reply]
Re: truncate a file from top by Random_Walk (Prior) on Nov 02, 2004 at 14:26 UTC
Reading the last X bytes is easy enough, an if your lines are fixed length this means the last N lines is equivalent # Generate some test data perl -e'for(0..9){@a=(A..Z,a..z," ",0..9);print$a[rand($#a+1)]for 1..8 +0;print$/}'>testdata # Grab the last 243 bytes == last 3 lines (80 chrs + \n) perl -le 'open F,"testdata";$a=243;seek F,-$a,2;sysread F,$data,$a;pri +nt$data;' gaNLTPyYjzh o0SFnuiNw18l9rpJvHXOcctK2gpLIZzODQou qal15OgfCEQsVGaQ3Ojmb +5h06pYM3x7 M0GgGwIdqqVmz1D5cWrOwC 0bzvp7bChgOtlzf1TE2 yh77hCIYhricePIH0VxiKHdv3Ye +a3zkN4aKZ6 sPBR0bryMtGTcGUk87B7UQmEpficHPy0rcjuwYTFyIbweuOcnxtyMpIm2BMcWJa30LVpFQ +5nLnJesQ4a # check it is true tail -3 testdata gaNLTPyYjzh o0SFnuiNw18l9rpJvHXOcctK2gpLIZzODQou qal15OgfCEQsVGaQ3Ojmb +5h06pYM3x7 M0GgGwIdqqVmz1D5cWrOwC 0bzvp7bChgOtlzf1TE2 yh77hCIYhricePIH0VxiKHdv3Ye +a3zkN4aKZ6 sPBR0bryMtGTcGUk87B7UQmEpficHPy0rcjuwYTFyIbweuOcnxtyMpIm2BMcWJa30LVpFQ +5nLnJesQ4a [download] update that seek should be sysseek to bypass buffering. Cheers, R.	[reply] [d/l]
Re: truncate a file from top by perlcapt (Pilgrim) on Nov 02, 2004 at 18:00 UTC
There is a handy module that will help: File::ReadBackwards. Just read as many lines as you want starting at the end of the file. It's a great time-saver when trying to find the last record/line/block/etc. in a really large file. perlcapt -ben	[reply]
Re: truncate a file from top by TedPride (Priest) on Nov 02, 2004 at 14:26 UTC
It would be easy if you just wanted to chop off the end of the file: `truncate('test.dat', 1024);` [download] But I assume since it's a log file that you want to chop off the beginning: `use strict; use warnings; my $fname = 'test.dat'; my $crop = 1024; my $handle; open($handle, $fname); $_ = (stat($handle))[7]; if ($_ > $crop) { seek($handle, $_ - $crop, 0); read($handle, $_, $crop); close($handle); open($handle, ">$fname"); print $handle $_; } close($handle);` [download] Of course, this method has problems too, in that additional lines could theoretically be appended to the file between the read and the write and therefore lost. File locking / unlocking probably isn't a viable solution, but you can just assume that up to x number of bytes might be written to the file between read and write, and edit your truncate accordingly: `my $crop = 1024; my $margin = 64; ... read($handle, $_, $crop + $margin);` [download] EDIT: zentara's solution solves that problem, and is much more elegant if you assume that the file will always be larger than the truncate size. ++	[reply] [d/l] [select]

use strict; use warnings; use diagnostics;

update