Fellow monks,

I need to put records into a storage container, and after a certain interval (configurable, some time between now and several days in the future) the record needs to be taken out and an action performed depending on the data it contains (e.g. send an email, update a database or log a message). The records will be relatively small(a few hundred bytes each) but there may be lots of them (up to hundreds of thousands). Records need to be stored persistently, to survive a server reboot or app failure. Records will be added by several independent processes, so the container needs to be accessible concurrently. Once the record has been acted upon it can be removed. The action does not have to take place at the exact time the interval is up, just after it has passed.

If it weren't for the persistency requirement I could run a network daemon which accepts incoming records, places them into a POE::Queue or a Heap and fires events when they are due. As it is, the best solution I can think of is to put records into an RDBMS, index on a field containing the due date/time, regularly query for select * from table where date >= now() and handle all records that are returned. This does not seem so great because it is possible the next query is issued when the last one isn't done dealing with the returned records (also, it'd be good if I could avoid being dependant on an RDBMS). It would seem ideal if I could just continually pop events off a heap, pausing if the next event in line hasn't reached it's due time. Does anyone know of a persistent heap implementation? I found Cache::File::Heap, but that uses DB_File internally and so will read/write the entire file into/from memory on every tie. Or is there a better solution for what I'm trying to achieve?

Thanks in advance for your time and advice