ginocapelli has asked for the wisdom of the Perl Monks concerning the following question:

Greatings and thanks for reading this! I run a free classifieds site and with the help of a friend, have added a few subroutines that create a new counter file that is named after the ad number (34.txt) and has a value of 0 to start. Now that all works fine, but ads posted prior to mods do not have a counter file. Rather than creating a new text file with a value of 0 for each ad (BIG JOB!), I thought I could write a script to do this for me. The script is supposed to:
  1. open the ads file
  2. flock it
  3. read each line into an array
  4. take just the first part of the array (the ad number), and create a new file for each, with a value of 0
  5. chmod each file to 0666
  6. close the ad file
  7. chmod it to 0666
  8. tell me that it is done
Here is the stuff:
#!/usr/bin/perl my $adfile = '/home/ebrae2/adskingston-www/cgi-bin/ebrae/data/ads.data +'; my $hits_dir = '/home/ebrae2/adskingston-www/cgi-bin/ebrae/hits/'; open(FILE, $adfile) || die "Can't open $adfile"; flock FILE, 2; while ($line = <FILE>) { chomp $line; my @counter_file = split(/\|/,$line); open (HITS_FILE, ">$hits_dir/$counter_file[0].txt") || die "Can't op +en $adfile"; ("$hits_dir/$counter_file[0].txt"); print HITS_FILE "0"; close (HITS_FILE); if ($os eq "unix") { chmod 0666, "$hits_dir/$$counter_file[0].txt"; } } close FILE; if ($os eq "unix") { chmod 0666, "$adfile"; } print "Content-type: text/html\n\n"; print "<BR><CENTER>"; print "<form><input type=button value=\"Success!\" onClick=\"locatio +n.href('http://www.adskingston.com')\"></FORM>";
I am new to this and know that there are probably some bugs. It does work, but only if there are 2 $$'s side by side on line 22, when I am telling it to flock each file it creates. AND, when I am connected via FTP (WS_FTP), run the script, and then try to open the "hits" folder, it doesn't show the new files, it hangs until I hit refresh... but it doesn't seem to be doing this for any other directories. Thought maybe it was just a fluke that the server was slowing down everytime I was trying to open that directory, maybe it is... the possibilities are endless!!!! Can someone tell me what I am doing wrong? Thanks! Edited by boo_radley for formatting

Replies are listed 'Best First'.
(bbfu) Re: using loop/array to write files
by bbfu (Curate) on Sep 09, 2001 at 22:09 UTC

    I think you may have had a problem while cut'n'pasting, since I don't see where you flock each of the new files and line 22 by my (vim's) count contains only the closing brace for the while loop. Perhaps you could repost the code, or even just the lines that are missing? Also, you seem to have a single element list in void context on the line immediately after the open (within the while loop). It looks like it might've been part of a function call but I'm not certain.

    Beyond that, I'm curious why you are locking the newly created files. Is this script supposed to be run while the server is active and other processes are possibly attempting to read these files? If the reading process has good enough error checking (ie, an empty file is assumed to be 0) then it probably shouldn't matter. *shrug*

    As for the FTP issue, how exactly are you executing the script via FTP? Is there a SITE command set up to execute programs? That sound like it might be a huge security hole. I'm not a security expert, however. As for the actual question (re: slowness), I don't really have any ideas. Perhaps if you post the missing lines...

    Ah well, those are my (unfortunately rather unhelpful) thoughts. Hope you can make something from them, and see if you can get those missing lines here and I'll take another look through. Good luck.

    bbfu
    Seasons don't fear The Reaper.
    Nor do the wind, the sun, and the rain.
    We can be like they are.

      I am not flocking each of the new files. I put the flock command after I open the first file. I thought locking the file is a good idea, since it is written to when a new ad is posted.

      I am not executing the script via FTP, I just have my FTP program open when I executed the script via my browser.

      And by line 22, I meant the line after "close (HITS_FILE);"

      I started the while loop with this line:while ($line = <FILE>) { and closed with a bracket just before: close FILE;

      Other than that, does the code look okay?
        The file lock does not good there. You have the following race:
        1. process 1 starts up and decides what file to create
        2. process 2 starts up and decides to create the same file.
        3. process 1 creates the file, and locks it.
        4. process 2 opens said file and tries to lock it.
        5. process 1 writes the file and exits.
        6. process 2 finally gets the file, overwrites what process 1 does and exits
        The fact is that while conceptually file locking is really simple, virtually nobody gets it right in practice. What you need to do instead is have a lockfile that always exists which, before doing operations that you want protected, you open and lock.

        That is the lock has to come before the process begins making any sort of decisions. And generally that means that you don't want to lock the file(s) you are going to work with.

Re: using loop/array to write files
by mandog (Curate) on Sep 09, 2001 at 22:26 UTC
    It sounds like the console of your local machine is more accessable than the logs of your server

    It might be a lot easier to debug your script if you were running it locally. ActiveState is very quick to install.

    I would download a set of 20 or so ads, change the value of $adfile and  $hits_dir to something that made sense on your local machine.

    Unless you have gigabytes of data to manipulate, I would even be tempted to create the files locally & then ftp them back up<P?
    --mandog

Re: using loop/array to write files
by mandog (Curate) on Sep 09, 2001 at 22:49 UTC
    Some possible problems

    I don't think there is a built in variable called  $os You may want $^O or if you have a  use English; at the start of your script, maybe  $OSNAME I think your chmod will never happen.

    You don't have a  use warnings and  use strict so perl didn't warn you that $os was neither declared nor initalized.


    --mandog

      Hi! Thanks for all the replies. I have taken some advice and tried to fix up the script, but I still can't use the "use strict;" line, I get an error.

      Also, when I execute the script, then try to open the foler in ws_ftp, it hangs and I have to hit the "Refresh" button in ws_ftp. Then the folder opens and the files are displayed. I am thinking that maybe the $adfile is not being closed?

      Here is the new code:
      #!/usr/bin/perl #use strict; can't use this, I get an error... my $adfile = '/home/ebrae2/adskingston-www/cgi-bin/ebrae/data/ads.data +'; my $hits_dir = '/home/ebrae2/adskingston-www/cgi-bin/ebrae/hits/'; $os = $^O; open(FILE, $adfile) || die "Can't open $adfile"; while ($line = <FILE>) { chomp $line; my @counter_file = split(/\|/,$line); open (HITS_FILE, ">$hits_dir/$counter_file[0].txt") || die "Can't cr +eate file"; ("$hits_dir/$counter_file[0].txt"); print HITS_FILE "0"; close (HITS_FILE); if ($os eq "unix") { chmod 0666, "$hits_dir/$counter_file[0].txt"; } } close(FILE); if ($os eq "unix") { chmod 0666, "$adfile"; } print "Content-type: text/html\n\n"; print "<BR><CENTER>"; print "<form><input type=button value=\"Success!\" onClick=\"locatio +n.href('http://www.adskingston.com')\"></FORM>";
      Can anyone see from looking at this code, what I am doing wrong? Thanks!
        Here is my new code:
        #!/usr/bin/perl use CGI::Carp qw(fatalsToBrowser); use strict; my $adfile = '/home/ebrae2/adskingston-www/cgi-bin/ebrae/data/ads.data +'; my $hits_dir = '/home/ebrae2/adskingston-www/cgi-bin/ebrae/hits/'; my $lines; my $counter_file; open(FILE, $adfile) || die "Can't open $adfile"; while ($line = <FILE>) { chomp $line; my @counter_file = split(/\|/,$line); open (HITS_FILE, ">$hits_dir/$counter_file[0].txt") || die "Can't cr +eate file"; ("$hits_dir/$counter_file[0].txt"); print HITS_FILE "0"; close (HITS_FILE); chmod 0666, "$hits_dir/$counter_file[0].txt"; } close(FILE); chmod 0666, "$adfile"; print "Content-type: text/html\n\n"; print "<BR><CENTER>"; print "<form><input type=button value=\"Success!\" onClick=\"locatio +n.href('http://www.adskingston.com')\"></FORM>";


        and it still doesn't work...