HDrider has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks. For some time now I have been quietly lurking and absorbing knowledge from the masters gathered here. I've used that knowledge to create a search engine for our library. Everything seems to work fine except when several people try to access it at once. Then they get the following error:

Software error: Cannot open h:/crc/cgi-bin/jsearch/database/files: at H:\crc\cgi-bin\jsearch\jsearch.cgi line 54.

The line in question is this:
tie %f_file_db, $dbm_package, $F_FILE_DB_FILE, O_RDONLY, 0755 or die "Cannot open $F_FILE_DB_FILE: $!";

The engine uses DB_File for database access.
The file(s)in question are files created by an indexing program.

I'm running Perl version 5.005_03 on an NT 4.0,IE 4.0, serv pak 6, server. It seems to be a rights issue but everything seems correct at the server level. Any ideas?

Hope I've given you enough info.

HDrider
Work to Ride and Ride to Work

Replies are listed 'Best First'.
Re: File rights?
by kschwab (Vicar) on Apr 30, 2001 at 19:42 UTC
    This topic is discussed at length in the "Safe ways to lock a database section" of the latest DB_File docs. A small excerpt:

    Safe ways to lock a database

Starting with version 2.x, Berkeley DB  has internal 
support for locking.  The companion module to this one,
BerkeleyDB, provides an interface to this locking 
functionality. If you are serious about locking Berkeley 
DB databases, I strongly recommend using BerkeleyDB.

If using BerkeleyDB isn't an option, there are a number 
of modules available on CPAN that can be used to implement
locking.

    The docs name these modules, as well as some alternative locking schemes. Good luck !

[reply]
      Thanks, I went and checked out these links. My problem is I
      I don't need to lock the files, they are read only when searched.

      What I need is for mutiple users to be able to search the db files at the same time.

      HDrider
      Never sneeze while wearing a full face helmet.

[reply]
        Locking isn't just for writes. If you read the links, you'll find mention of creating multiple copies for readers. Also, the BerkelyDB module is mentioned, which supports the single writer/multiple reader scenario you are describing (without making copies).

        Not to be terse, but go back and read it again :)

[reply]
Re: File rights?
by astanley (Beadle) on Apr 30, 2001 at 19:14 UTC
    Just a thought - as far as the indexing program goes - the index program has to open the DB for write...AFAIK WinXX doesn't like this situation (a file is opened for writing and another process also attempts to open it). There may be some DB_File specific issues here but I am not positive of the file-locking policies of DB_File. One solution for you is to do a manual file-locking situation. While the DB is open create a temporary .lock file somewhere. Once you close the DB remove the file. While the .lock is present have your other processes sleep for one second and check for it again. This would allow you to setup a manual queue. Using a DB Server would also solve the problem - let the server handle simultaneous requests.

    -Adam Stanley
    Nethosters, Inc.
[reply]
      OK. This is a bit over my head/skill level right now :)
      The indexing program runs each weekend when there is no one around,
      it hasn't given me any problems other than speed but it is chewing it's
      way through 4 gigs of files.

      What I'm trying to figure out is how to allow multiple users to search the index created
      DB files at the same time. I don't need the files locked.

      Thanks for the reply though, now I have something else to try and learn.

      HDrider
      A cold hamburger can be reheated quite nicely by strapping it to an exhaust pipe
      and riding forty miles :)

[reply]
Re: File rights?
by tilly (Archbishop) on May 01, 2001 at 01:46 UTC
    I am going to go out on a limb and say that your problem does not involve permissions at all and is due to the issue that I mentioned at Re (tilly) 3: BerkleyDB versions 2.x , 3.x, which is that the underlying Berkeley DB libraries cannot be accessed simultaneously from multiple machines due to limitations in how shared memory is implemented.

    Furthermore I am going to guess that this error comes from inside the C library, and is not (usefully at least) trapped by DB_File, which is why $! is not populated.

    Using locks so that people don't really access this simultaneously is likely to help. (I cannot guarantee that though.) Another solution for read-only access is to have people operate off of local copies of the index files.

    And an incidental note for the casual reader. The underlying shared memory issue is one of the reasons why at Re (tilly) 2: Synchronizing variables (in mod_perl) across Apache instances? I said I prefer avoiding introducing shared memory when it is not yet needed. Berkeley DB has little choice, among other things they implement their locking subsystem between processes in a shared memory segment. However if you are building something that will ever potentially be shared across machines, avoid it up front.

[reply]

Back to Seekers of Perl Wisdom