in reply to How to reverse a huge file in Perl?
Update Wed Mar 26 10:05:03 CET 2008: Added number of lines in input file.# ls -lh /var/lib/mysql/ibdata1 -rw-rw---- 1 mysql mysql 66M Mar 19 11:05 /var/lib/mysql/ibdata1 # wc -l /var/lib/mysql/ibdata1 323833 /var/lib/mysql/ibdata1 # time tac /var/lib/mysql/ibdata1 > /tmp/bar real 0m0.643s user 0m0.173s sys 0m0.322s # time /usr/bin/perl > /tmp/foo use warnings; use strict; use File::ReadBackwards; my $bw = File::ReadBackwards->new( '/var/lib/mysql/ibdata1' ) or die "$!" ; while( defined( my $log_line = $bw->readline ) ) { print $log_line ; } __END__ real 0m29.076s user 0m11.910s sys 0m10.819s # md5sum /tmp/foo 2b31d9f47525853842d5dbce584bd95c /tmp/foo # md5sum /tmp/bar 2b31d9f47525853842d5dbce584bd95c /tmp/bar
Update Wed Mar 26 10:11:31 CET 2008: It seems that the tac + process file forwards using Perl approach combined is still much faster than process file using Perl and File::ReadBackwards:
# time /usr/bin/perl > /tmp/foo use warnings; use strict; use File::ReadBackwards; my $bw = File::ReadBackwards->new( '/var/lib/mysql/ibdata1' ) or die "$!" ; while( defined( my $log_line = $bw->readline ) ) { $log_line =~ s/1/2/g; print $log_line; } __END__ real 0m27.431s user 0m12.906s sys 0m10.701s [root@afflinux aff]# time /usr/bin/perl > /tmp/bar use warnings; use strict; my $FH = undef; open($FH, '/var/lib/mysql/ibdata1' ) or die "$!" ; while( my $log_line = <$FH> ) { $log_line =~ s/1/2/g; print $log_line; } __END__ real 0m5.249s user 0m1.535s sys 0m0.293s
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: How to reverse a huge file in Perl?
by BrowserUk (Patriarch) on Mar 26, 2008 at 09:02 UTC | |
|
Re^2: How to reverse a huge file in Perl?
by samtregar (Abbot) on Mar 26, 2008 at 17:02 UTC | |
by BrowserUk (Patriarch) on Mar 26, 2008 at 17:48 UTC | |
by samtregar (Abbot) on Mar 26, 2008 at 20:00 UTC | |
by BrowserUk (Patriarch) on Mar 26, 2008 at 20:56 UTC | |
by Anonymous Monk on Mar 26, 2008 at 18:48 UTC | |
by BrowserUk (Patriarch) on Mar 26, 2008 at 19:39 UTC | |
|
Re^2: How to reverse a huge file in Perl?
by Arunbear (Prior) on Mar 27, 2008 at 00:00 UTC |