toonski has asked for the wisdom of the Perl Monks concerning the following question:

I've been trying to do this thing with inserting and deleting stings of data into files. i'm writing a flatfile database with fixed length records and i wrote a routine that inserts strings of data into a file, but every time i do it it pads null strings at the end of the file. is there just a standard module that can do this for me faster and easier? deleting would help as well. thanks

Replies are listed 'Best First'.
(jeffa) Re: inserting/deleting file data
by jeffa (Bishop) on Mar 23, 2003 at 15:55 UTC
    I personally would rather use DBD::SQLite for such a problem, but it sounds like you are using pack, no? If that is the case, are you packing/unpacking with 'a' or 'A'? 
    my @data = (
  'thrakb    boom ',
  'a d j k l j s f',
  '012345678901234',
);

foreach (@data) {
   print 'a unpack: ', join('|', unpack('a5a5a5',$_)), "\n";
   print 'A unpack: ', join('|', unpack('A5A5A5',$_)), "\n";
}

my @record = (
   [qw(123 blah 21)],
   [qw(45 sadflkj 9)],
   [qw(4532 j 99)],
);

foreach (@record) {
   print 'A pack: ', pack('A4 A10 A2',@$_),"\n";
   print 'a pack: ', pack('a4 a10 a2',@$_),"\n";
}

    [download]
    Try packing with 'a' instead of 'A'. Note that packing with 'A' will add trailing whitespace: 
    
           a                         A
  pack  does not add space     adds trailing space
unpack  does not remove space  removes trailing space
    Update: added pack example, prolly what i should have used instead of the unpack example. ;)

    jeffa

    L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)
[reply]
[d/l]
Re: inserting/deleting file data
by BrowserUk (Patriarch) on Mar 23, 2003 at 16:10 UTC

    You might find that Tie::File in conjunction with pack might make your life simpler. You need to select you pack formats very carefully to match the format of your data.

    Examine what is said, not who speaks.
    1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
    2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
    3) Any sufficiently advanced technology is indistinguishable from magic.
    Arthur C. Clarke.
[reply]
      Agreed with BrowserUk. Tie::File is the most ideal in this case.

      As Tie::File hides the physical layout at some level (the layout among lines, obviously you still need to worry the internal structure of each record, which is your pack), and now each line of the file is presented as an element of an array.

      I had a post analyzing how Tie::File uses memory, and you would see that Tie::File's author used memory in a very smart way.
[reply]
Re: inserting/deleting file data
by Limbic~Region (Chancellor) on Mar 23, 2003 at 15:59 UTC
    toonski,
    I see that you have been at the Monastery a while, but haven't done a lot of posting. It might be helpful to spend some time on how reading the site FAQ for pointers on how to ask a question effectively. Without any code, it is almost impossible to help you. Some questions for you:

  • Are you using pack/unpack correctly to ensure your fixed length records are always of the correct length?
  • Is there a reason you can't use a DBM module or is there a specific reason for the flat file fixed length records?
  • Are you using open, seek, and tell to find record locations or sysopen, sysread, and sysseek

    More than likely, you can get away with the CPAN module I listed above, but if you want to do a flat file fixed length database, please provide your code.

    Cheers - L~R

    Update: How is it when you start to post to a reply, no one else has done so, but by the time you click submit - 6 other monks have provided better solutions before you? I would still like to see the code even if Jeffa's crystal ball idea was spot on - it will help other monks on their journey.

[reply]
Re: inserting/deleting file data
by vek (Prior) on Mar 23, 2003 at 16:14 UTC
    You've already received some good suggestions about using DBD::SQLite so I thought I'd point out an alternative. When working with flat files, I've used DBD::CSV in the past with some success. Admittedly this was before I discovered DBD::SQLite and would probably go with that instead - YMMV of course so feel free to check them both out. I also notice from browsing the CPAN that there's a module called DBD::Sprite although I must admit that I've never used it myself.

    -- vek --
[reply]
    ...
    by toonski (Beadle) on Mar 23, 2003 at 16:18 UTC
      eep. i wrote a giant reply explaining my situation, but right before i posted i saw that someone had mentioned that tie::file can insert and delete data.

      i'm using pack, and correctly (the db is already made). it's inserting/deleting the data that was killing me.

      thanks everyone :)
[reply]
Re: inserting/deleting file data
by robartes (Priest) on Mar 23, 2003 at 15:51 UTC
    I think you're going to have to post some code for us to be able to help you. Most of us aren't psychics.

    If you post the part of the code that writes to the flatfile, people can look it over and report anything that might be generating those null bytes at the end of your file.

    Update: Judging by the replies below, I apparently underestimated the number of psychics around here ;). jeffa's hunch wrg pack might well be spot on.

    CU
    Robartes-

[reply]

Back to Seekers of Perl Wisdom