I did something like this once to speed up changes to the password file. Many simultaneous writers were causing too much of a delay, much like you're seeing.

I prototyped a system where, in essence, if the password file was locked the writer would write to a log instead, and whenever a writer got ahold of the lock, it would make its own changes along with any changes in the logfile. It was more complicated than that, using locking and atomic filesystem operations to prevent race conditions and deadlock, but it worked, and provided a huge speedup. Essentially, this log file acted as what database folks call a log and what filesytem folks call a journal. You could get this same effect more simply by having all writers write to a logfile, and running an updater from cron periodically.

There were two reasons why this worked. First, in our situation it was OK for write requests to be delayed for a few minutes, and for future read requests to not see the writes right away. Otherwise it would have involved modifying all readers to look in the log first, and we would have lost any speed advantages. Second, updating the password file with many changes was about the same speed as for a single change, so batching the requests made a significant speed improvement. If either of these isn't true for your application, you'll find it much harder.

Perhaps tellingly, we ended up scrapping the prototype and just buying a faster system.