ghopper has asked for the wisdom of the Perl Monks concerning the following question:

Apologies in advance, I am a Perl and programming novice. This is probably a Stupid Question, but I've looked for an answer in the archives and I still can't figure it out.

I have a CGI script that (1) writes to a datafile, then (2) immediately reads that file in order to print to a webpage. The problem is that the changes I make (deletions, actually) do not "take" immediately. For example, if I delete record #150 (by using splice--backwards, if there's more than one record being deleted), the page refreshes and record #150 is still on the page (even though it's gone from the datafile). If I reload the page (not, in case you're wondering, by resubmitting the form data, which would try to re-delete the record), then and only then will record #150 be gone from the webpage.

Now here's the weird part (to me): this happens only with the last three records. Suppose the last three records are 'A', 'B', and 'C'. For each of the last three records, a different thing happens if I try to delete it:

If I delete record $array[-1], then on the webpage, nothing happens until I refresh (without resubmitting). Then and only then the record (in this case, 'C') disappears.

If I delete record $array[-2], or 'B', then on the webpage, the last three records read: 'A', 'C', 'C'. THEN if I refresh (without resubmitting), the last two records are (as originally expected) 'A' and 'C'.

Finally, if I delete record $array[-3], or 'A', then on the webpage, the last three records read: 'B', 'C', 'C'. And again, if I refresh, the last two records are instead, as originally expected, 'B' and 'C'.

Whenever I delete any of the records in the beginning or middle of the file (before the $array[-3] position), the item(s) immediately disappear(s).

My only thought after research is that it might have something to do with a needed file lock, but I don't know anything about that, and it sounds implausible anyway. (The datafile is written-to and closed before it is reopened for reading.)

Please expose what embarrassing error I have committed! Or, if you need more info, let me know.

--ghopper :-)

Replies are listed 'Best First'.
Re: Datafile doesn't update before being read
by George_Sherston (Vicar) on Jun 06, 2002 at 19:36 UTC
    Good that you post some code. I don't entirely agree with respected sibling Sifmole - I'd encourage you not to be parsimonious with your code posting. As it was, your question was about a thing that happens when you refresh your web page, if I understood it, but the code you posted doesn't seem to cover that.

    So my first suggestion is, don't be shy, post away (on the other hand other monks may be working with a more current version of SOPW::Psychic than I have and may be able to answer based on what we have).

    My second suggestion is - you say

    I have a CGI script that (1) writes to a datafile, then (2) immediately reads that file in order to print to a webpage.

    - now there's probably a good reason for this, but on the face of it, why bother with the read at (2)? - you've already got the data in memory, so couldn't you just send it to the browser?

    § George Sherston
      I guess I was trying to keep my program "modular," reduce the number of global variables to a minimum, so I guess that's why didn't just use the array I made earlier later.

      In retrospect, however, you're 100% right. I simply made @strmeta a global variable and didn't bother reprocessing the data. Then the problem went away and as an extra bonus, the page loads faster.

      Thanks a lot, George!

      --ghopper

        Your'e welcome. I (and others doubtless) applaud your desire to be modular. How about this, then?: take your subroutine for printing output from an array. Not sure what you're calling this - say print_page. Now, instead of making @strmeta a global, make a local variable and pass a reference to it to the subroutine.

        That is, call your subroutine like
        my @strmeta = ..... [...] print_page(\@strmeta);
        Then in the subroutine make the first line
        my $strmeta = shift;
        And thereafter, where before you had @strmeta use @$strmeta, and where you had $strmeta[$foo] put $strmeta->[$foo] - which will "de-reference" (i.e. get the data out of) your reference $strmeta.

        Then, if you set it up like that, you can call the same subroutine from two different points in your script - (a) when you've created @strmeta from your form input, and (b) when you've created it from retrieved data. Your subroutine won't care where the data comes from, so long as it comes in an array reference.

        For your last trick, put print_page() in a module, and use it in all your other scripts!

        Enjoy!

        § George Sherston
Re: Datafile doesn't update before being read
by Sifmole (Chaplain) on Jun 06, 2002 at 20:02 UTC
    Are you by any chance reading the changed file back into the array that you read the original into?

    My theory is this: You have an array that holds the original file data; you then create some array which you modify via the splice. You then output the keys that you want to keep. You then read in the now modified file into the original array that you started with via some method that is not clearing the array before you read into it.

    read from file to @orig @orig = [ 1 2 3 4 A B C ] @mod_copy = [ 1 2 3 4 A B C ] splice @mod_copy @mod_copy = [ 1 2 3 4 A C ] print join of @mod_copy to file file is now ( 1 2 3 4 A C ) read from file to @orig by indexing @orig = [ 1 2 3 4 A C C ]
    Notice how the last C is leftover? The reason I suspect this is because the results you report look like the original list overwritten with the new list, but not erasing the originals that are left beyond the end of the list.
      Something like this has to be right. I'll go check. Thanks a bunch, I'll report back!
      You had to be right about this (your explanation was the only one I had and very elegant, to boot), but I couldn't spot where I was reading the changed file back into the array that you read the original into. I thought (could be wrong) I had made a local copy of the @strmeta ("structured metadata") variable for purposes of the deletion bits. I still think you had to be right, but I couldn't find the exact source of the error.

      Instead I took George's advice and simply made @strmeta a global variable (er, well, I don't actually know if "global" is the right word--I didn't make a local copy), and didn't reload it after writing it to file, just used the altered @strdata array to print out the changed data. The problem went away!

      Thanks very much to everyone!

      --ghopper

Re: Datafile doesn't update before being read
by Aristotle (Chancellor) on Jun 06, 2002 at 19:04 UTC

    You may need to turn buffering off on the filehandle you're using to write to the file. Also, if it weren't for other records immediately disappearing I'd suspect you don't truncate the file in which case the last record would stick around - but then it would have to happen for all records and wouldn't disappear on next refresh..

    I agree, we need some code here before we can give any sort of actual advice.

    Makeshifts last the longest.

Re: Datafile doesn't update before being read
by Popcorn Dave (Abbot) on Jun 06, 2002 at 19:38 UTC
    I may be able to shed a little light on what you're attempting to do. A while back I wrote a CGI app for a friend's web site that allowed them to update items for sale solely from a web page. It was set up as a 2 frame web page with an add items section on the left, and items for sale on the right. The idea was to do a very simple database for a web page that was built on the fly.

    Items could be added via a form on the left and the items then were updated on the right.

    In addition, the items on the right were displayed with a check box and that part was set up as a form as well so that multiple check boxes could be selected, and deleted all at once. When an item was deleted, the script called itself again so as to display the new data. That may be one option for you to consider in your program.

    Hope that helps!

    Some people fall from grace. I prefer a running start...

      Yep, I'm doing something very similar. Basically, I've got a list of weblinks with checkboxes next to them. The function I'm working on simply allows the user to check off a bunch of boxes at a time, press "delete," and the selected links are deleted. The page calls itself.

      I guess it would have been helpful to say that up front!

      --ghopper

Re: Datafile doesn't update before being read
by Sifmole (Chaplain) on Jun 06, 2002 at 18:37 UTC
    The relevant piece of code would help greatly in being able to get advice. Attempt to reduce the required code to the smallest required amount to exhibit the problem, and post that.
      Thanks for the reply. I was afraid of that. The script is kind of long, and I'm not sure what is best thought relevant, but here we go. '$file' refers to the datafile I'm working with.
      my $do; # function # Get delete_links input (which links should be deleted?) if ( $input =~ m/Delete\+all\+checked\+links%21$/ ) { $do = 'delete'; $input =~ s/(.*?)&delete_links=.*$/$1/ ; @del_links = split /&/, $input; @del_links = map { s/(\d+?)=checked/$1/ ; $_; } @del_links; @del_links = reverse @del_links; # so that last ones are done f +irst } [...] my @metadata = &build_metadata_array; my @strmeta = &parse_metadata(@metadata);

      I am hoping you won't need to see the latter subroutines. The first simply reads in a datafile ($file), the second creates an array of references to the split data records. So, for example, @{$strmeta[0]} might consist of ('ap.1', 'ap.2', 'nyt.4', 'some random text about those ids').

      if ($do eq 'delete') { foreach my $num ( @del_links ) { splice (@strmeta, $num, 1); } } [...] # Rewrite metadata/$file open (META, ">metadata/$file"); foreach my $rec ( @strmeta ) { my $summary = pop @{$rec}; my $ids = join (/,/, @{$rec}); print META "$ids\n$summary\n\n"; } close META;
      Is that enough?
        Well, I don't know why it's failing (you didn't show us the code that reads the data for instance), but I do know this is going to fail if you are getting two requests in a short time. The first thing that happens when you open the file for write is that it's getting truncated. What if another instance is reading from the file? What if another instance is also writing? Your data will get corrupted. Also, you open the file, and you don't check the return value. What if the file cannot be opened? Same for the close, you might get an error here (disk full, for instance). You don't check.

        Abigail