in reply to Removing the first record in a file containing fixed records

Not really without altering the structure. You can truncate files from the end, but you can't . . . precate (if I may coin a terrible word) things off the front. A kludge (although this is remotely similar to what some databases do) is to alter some part of the record to signal it as invalid (perhaps setting a field to say XXXDELETEDNOUSEXXX) and then have your downstream processing ignore these records; periodically you can have a utility "vacuum" out the deleted records by copying the live records into a new file as you mention above. Alternately if ordering of records in the file is not important you can overwrite deleted records with new ones in place (perhaps maintaining an external "free list" of offsets of deleted record slots).

Of course doing this type of thing you're already on your way towards reimplementing your own database system so you might want to consider taking that plunge and letting someone else do the heavy lifting for you and concentrate on your processing tasks.

The cake is a lie.
The cake is a lie.
The cake is a lie.

  • Comment on Re: Removing the first record in a file containing fixed records

Replies are listed 'Best First'.
Re^2: Removing the first record in a file containing fixed records
by Crackers2 (Parson) on Jul 18, 2008 at 00:48 UTC
    If ordering is not important he could just read the very last record, overwrite the first record with it and truncate the file before the last record.
[reply]
      Since sparkle asked for an efficient method that works even on large files (2GB is sometimes the next limit), I agree with Crackers2 suggestion that will work well since it preserves disk space and implies a minimal amount of copy operations. But in case the header has a different size than the other records, order is important, or in case a copy of the file is still required, I would suggest to give that task away to a tool that is optimised for that kind of operations. Where available, a system(...)-call to tail or dd might be worth consideration.
[reply]
[d/l]
[select]
Re^2: Removing the first record in a file containing fixed records
by goibhniu (Hermit) on Jul 18, 2008 at 17:27 UTC

    I think that word is decapitate.

    #my sig used to say 'I humbly seek wisdom. '. Now it says:
    use strict;
    use warnings;
    I humbly seek wisdom.
[reply]

In Section Seekers of Perl Wisdom