Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello. I'm having a problem updating or deleting certain records in a flat file. Right now, I present an HTML form that displays all the records(lines) in my flat file. Next to each line I have a radio button which holds a unique number as its value. In my flat file, records have a record number at the end of it. When a radio button is selected and either an edit or delete button is pressed, how can I edit, delete the corresponding record in my file?
  • Comment on editing/deleting record in flat file database

Replies are listed 'Best First'.
RE: editing/deleting record in flat file database
by chromatic (Archbishop) on Oct 17, 2000 at 23:55 UTC
    Another approach would be to create a Perl data structure out of the file -- just once -- and then serializing it to disk with something like Data::Dumper or Storable.

    Updating it would be as simple as manipulating a normal array or hash. You'd have to add serialize/unserialize steps at the start and end of your program, but that's easy to do.

    As the number of records increase, so will the memory usage. Of course, the hassle of updating a flat file increases with the current technique, so at some point you might consider a true database. (Somewhere after 100 or 150 records is my personal guideline.)

Re: editing/deleting record in flat file database
by merlyn (Sage) on Oct 17, 2000 at 21:58 UTC
    You can use DBD::RAM to "edit" the file using the normal DBI interface. Very slick. You can even convert it from flat file to CSV or XML or even easily copy it to a real DB backend.

    -- Randal L. Schwartz, Perl hacker

Re: editing/deleting record in flat file database
by turnstep (Parson) on Oct 17, 2000 at 23:47 UTC

    Here's a way to do it if your flatfile consists of one record per line, with whitespace before the "record number" at the end:

    #!/usr/bin/perl -- -*-fundamental-*- use strict; use Fcntl qw(:flock); ## Set these as needed... my $flatfile = "flat.txt"; my $num2delete = "12"; my $result = &DeleteRecord($flatfile, $num2delete); if ($result) { print "No records were deleted.\n"; } else { print "Deleted record number $num2delete.\n"; } exit; sub DeleteRecord($$) { my $flatfile = shift; my $num2delete = shift; ## Open the flatfile in read/write mode open(FLATFILE, "+< $flatfile") or die "Could not open $flatfile: $!\ +n"; ## Lock it exclusively flock(FLATFILE, LOCK_EX) or die "Could not lock $flatfile: $!\n"; ## Try and find a match my $found=0; while(<FLATFILE>) { last if /\s+$num2delete$/ and ++$found; } ## If no match, we clean up and return a 1 unless ($found) { close(FLATFILE); ## Also unlocks for you return 1; ## Have the caller handle this case; } ## Mark the spot where the record to be deleted starts my $here = tell(FLATFILE) - length $_; ## Read in the rest of the file: my @slurp = <FLATFILE>; ## Rewind to the spot we marked earlier seek(FLATFILE,$here,0); ## Stick everything else back in, overwriting the file as we go: print FLATFILE @slurp; ## If we don't truncate, the last record will appear twice (think ab +out it) truncate(FLATFILE, tell); ## Close and unlock, and return success to the caller close(FLATFILE); return 0; }
RE: editing/deleting record in flat file database
by Adam (Vicar) on Oct 17, 2000 at 22:20 UTC
    Did anyone else cringe at the phrase "flat file database"???
    I know I did.

    Its one thing to toss data around in flat files. (In fact, appending to flat files is a great way to collect data for parsing.) But flat files are NOT data bases. There is a reason for fifty million variations of SQL DBs. If you are doing something that works like a database, by all means... use a data base!

      Errrr, I didn't.

      It is a matter of scale. If the amount of data is small and the database design is simple then why not?
      Least ways you can spare yourself the great overhead (and often cost) of creating a SQL database and having to support it (I've seen so many sites where whole teams of DBA's were needed to support relatively simple databases)

      For under 5000 records I'm not convinced that opening a database connection squirting your SQL across the TCP socket and getting a reply back is any quicker than scanning the file. Especially if your data set has no suitable keys. Having said all that I'm quite a fan of AnyDBM_File as well

      Maybe I'm just an old disillusioned DBA :-)

      "We are all prompted by the same motives, all deceived by the same fallacies, all animated by hope, obstructed by danger, entangled by desire, and seduced by pleasure." - Samuel Johnson
Re: editing/deleting record in flat file database
by AgentM (Curate) on Oct 17, 2000 at 23:02 UTC
    If you insist on using a "flat file", which I assume means line-delimited records in an ordinary file, then the fastest (programmers') way to delete or insert a record is to read the file into an array (Perl will read it automatically into your array with line-delimiters as default), switch it in your script with the whole file in MEM and rewrite the file. Of course, this is not the optimal solution. The optimal solution involves moving (on average) 50% of all data sets in the file to delete and none to append. To delete, find the line where you want the deletion to begin/end and copy the rest of the file "up". Another solution would be to comment it out in some manner and "optimize" you crappy database on a cron cycle by deleting comments from the file every so often. Edit is yet more complicated since it may involve reducing or enlarging the file depending on how much input is given. Of course, as the others said, this is very strange limiting situation and it would be extraordinarily easy to convert your text file to a form of AnyDBM_File, which is the recommended solution if you're not running a dbengine. Since this is CGI, as you mentioned, you'll want to be careful what the user passes you- perhaps max line lengths, valid chars, etc.
    AgentM Systems nor Nasca Enterprises nor Bone::Easy is responsible for the comments made by AgentM.
Re: editing/deleting record in flat file database
by Anonymous Monk on Oct 17, 2000 at 22:28 UTC
    Regardless of the fact that I'm using a delimited text file, is there a quick hack to do this ( also without the help of DBD::RAM )? I'm still learning.