Or is my crystal ball manfunctioning again? I (too) hate it when that happens.

Sorry, but a better answer regarding your "corruption" problems is unlikely to surface until we see the code... and the evidence (error messages, warnings, output?) of the issue.

As to the first part of your note, for starters, you'll do well to Super Search for "Storable" and see what prior discussion is stashed in the Monastery's nooks and crannies.

Update: Correcting typo (s/seem/see/ in previous para

      and -- in light of OP's reply below -- "I didn't include any code because I don't believe it is the code itself... (man, how many times have we all heard that?) :)"

FW(little?)IW, if the problem is indeed simultaneous write attempts, I doubt your honeypot's functionality would be crippled by:

1. providing locking on your target
2. writing a procedure to buffer the second write until the lock is released

Note that the doc does appear, quite clearly, to outline a procedure for locking:

ADVISORY LOCKING

The lock_store and lock_nstore routine are equivalent to store and nstore, except that they get an exclusive lock on the file before writing. Likewise, lock_retrieve does the same as retrieve, but also gets a shared lock on the file before reading.

As with any advisory locking scheme, the protection only works if you systematically use lock_store and lock_retrieve. If one side of your application uses store whilst the other uses lock_retrieve, you will get no protection at all.

The internal advisory locking is implemented using Perl's flock() routine. If your system does not support any form of flock(), or if you share your files across NFS, you might wish to use other forms of locking by using modules such as LockFile::Simple which lock a file using a filesystem entry, instead of locking the file descriptor.

Now, I don't use storable, and would have to agree (caveat: I struggle with documentation often, and only slightly less often come to an understanding which is wrong) that Storable's doc might benefit from somewhat more exposition and several more examples.

However, the doc DOES discuss the Storable::is_storing and Storable::is_retrieving methods in its "Predicates" section... and that, for a WAG, may be relevant to your suspicion that errors occur when "two set requests might come in at the same time against the same target and try to write to the very same persistent data." (and which frankly gives your remarks a rather strong code-smell.

And with that, recurse to the suggestion above: that better help will be more likely forthcoming when there's some code to examine...and info on the evidence of corruption!

ww,

I went with your first suggestion of locking and it indeed worked like a charm. My original train of thought was that it would be better if I checked the status to see if the file was in the middle of storing, and if it was, just throw out the secondary write requests because it is all fake data to begin with.

That turned out to be a bad idea any way. My script isn't written in a way that could properly use the is_storing method in the first place so that would have got me no where.

With the simple change of replacing 'store' to 'lock_store', all the corruption has faded away and to me, the code now smells like roses. :)

I appreciate you taking the time to help me out.

-Dave

ww, thanks for the response. I didn't include any code because I don't believe it is the code itself... (man, how many times have we all heard that?) :) ...however the code would benefit if I could tell if the persisted data is already being written too. What I have is a honeypot which forks my PERL script to simulate SNMP agents. Occasionally, two set requests might come in at the same time against the same target and try to write to the very same persistent data which I believe is where the corruption happens. I can't really avoid that and feel I need to incorporate these methods to avoid concurrent store requests against the same data.