Is Cache::Cache process safe? (process was thread)

talexb has asked for the wisdom of the Perl Monks concerning the following question:

I'm seeing problems in Cache::Cache when I have multiple instances of a script running on a grid engine trying to cache identical objects at (more or less) the same time. The error I get is

Can't use an undefined value as an ARRAY reference at
/usr/lib/perl5/site_perl/5.8.0/Cache/FileBackend.pm line 586
[download]

I presume that this is the result of a race condition.

Any thoughts? I've done a Super Search on this, going back halfway through the database, and didn't turn up anything useful.

Alex / talexb / Toronto

"Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

Update At about 1335EDT, requested moderation to change all titles from 'thread' to process. Don't know what I was thinking -- too soon after lunch, perhaps.

Comment on Is Cache::Cache process safe? (process was thread) Download Code

Replies are listed 'Best First'.
Re: Is Cache::Cache process safe? (process was thread) by merlyn (Sage) on May 08, 2006 at 22:57 UTC
I worked with the developer of Cache::Cache to ensure that it was safe working on one machine when it was originally published, but since you mention NFS later in this thread, I'd say all bets are off for NFS. -- Randal L. Schwartz, Perl hacker Be sure to read my standard disclaimer if this is a reply.	[reply]
Re: Is Cache::Cache thread safe? by perrin (Chancellor) on May 08, 2006 at 16:49 UTC
Are you actually using threads, not processes? It may make assumptions about the current working directory. It is safe between multiple processes. What's on your line 586? In my copy, that line is setting a umask.	[reply]
Re^2: Is Cache::Cache process safe? (process was thread) by talexb (Chancellor) on May 08, 2006 at 17:01 UTC
Umm, I think I have been using the word 'threads' in the wrong context, sorry. I have a bunch of grid engine nodes running scripts that use Cache::Cache on an NFS-shared directory tree, and my hypothesis is that two jobs are trying to add two identical objects to the cache at roughly the same time. So I guess that's two processes, not two threads. And line 586 is the return statement in the following subroutine: `sub _get_key_for_unique_key { my ( $self, $p_namespace, $p_unique_key ) = @_; return $self->_read_data( $self->_path_to_unique_key( $p_namespace, $p_unique_key ) )->[0];` [download] Alex / talexb / Toronto "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds	[reply] [d/l]
Re^3: Is Cache::Cache process safe? (process was thread) by perrin (Chancellor) on May 08, 2006 at 17:15 UTC
You may want to check that you have the latest release, since there have been changes to this code in the last two releases. Many people use this module with web apps which are multi-process, so it has been pretty well tested for race conditions. Your error message implies that Build_Path returns undef for you. I'm not sure why that would happen, but you may want to add some extra debugging to that method. Alternatively, you could ditch this approach. It's going to be quite slow using Cache::FileCache over NFS. You'd get much better performance caching this data in a simple MySQL table.	[reply]
Re^4: Is Cache::Cache process safe? (process was thread) by talexb (Chancellor) on May 08, 2006 at 17:23 UTC
Re: Is Cache::Cache process safe? (process was thread) by cowboy (Friar) on May 08, 2006 at 18:35 UTC
While this doesn't really answer your question, you might consider using Cache::Memcached rather than Cache::Cache with the file backend via NFS. You get the shared network based cache without having to worry about any potential problems dealing with file locking, nfs, and it should perform much faster as well (assuming your cache data can fit in the ram of 1 or more machines)	[reply]