http://qs1969.pair.com?node_id=1039696


in reply to Edit huge file

Alright, thanks all; without a method to do this more cleanly I can just write my own program to read the files, rewrite them without the lines, etc. etc..... A little disappointed there's not a better way to do this, oh well though.

Replies are listed 'Best First'.
Re^2: Edit huge file
by roboticus (Chancellor) on Jun 19, 2013 at 02:46 UTC

    AI Cowboy:

    If you just want information on the first line to disappear and don't really have any particular reason that you must move the data in the file, you could always overwrite it with blanks. You could do that *very quickly*:

    #!/usr/bin/perl use strict; use warnings; use autodie; # Open file in read/write mode open my $FH, '+<', 'tmp.txt'; # Skip the first line my $t = <$FH>; # Remember the starting location of the second line my $pos = tell $FH; # Read the second line $t = <$FH>; # Rewind back to the start of the second line and obliterate it seek $FH, $pos, 0; print $FH "*" x (length($t)-length($/));

    Here's a quick demonstration:

    $ head -5 tmp.txt You are on the edge of a breath-taking view. Far below you is an active volcano, from which great gouts of molten lava come surging out, cascading back down into the depths. The glowing rock fills the farthest reaches of the cavern with a blood-red glare, giving every- thing an eerie, macabre $ perl obliterate_second_line.pl $ head -5 tmp.txt You are on the edge of a breath-taking view. Far below you ************************************************************ come surging out, cascading back down into the depths. The glowing rock fills the farthest reaches of the cavern with a blood-red glare, giving every- thing an eerie, macabre

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

      I used software that had its "database" in a set of text files, and that is how it removed data, awaiting a repack. It would identify the record as a deleted record, record the length, then fill the rest with some padding character of some sort to remove the old data.

      --MidLifeXis

Re^2: Edit huge file
by davido (Cardinal) on Jun 19, 2013 at 01:27 UTC

    Keep in mind that this isn't a Perl shortcoming. It's how files work, across any language, and across every operating system I've used (which is certainly not every, but a large enough sampling to see a trend. ;)


    Dave