in reply to maximum number of open files

I had a similar problem once (long time ago) and I cheated. It was an exceptional condition - most of the time I was not getting close to the limits.
I abstracted the IO part of the application (I was lucky to have a layered design which made it easy) and put that as a separate module. When I got close to the limit I created another process which just ran the IO. I communicated with this "IO Server" using named pipes, but any appropriate IPC could be used. I could just keep creating "IO Server" processes as I needed.
There are performance implications, but they were not noticable. However it was a lot of work, far better to avoid using huge numbers of opens.