in reply to WHY copying does happen (fork)

What you'd like to have, and I don't know if you can have, is either "several true-threads, all of which therefore operate in one memory-space, or you want a data-structure that exists in a single shared memory-space that is truly attached-to by everyone. I don't know how you do that in Perl.

Basically, what you're trying to achieve here is a read-only shared data-store, and I have no doubt that it is the reference-counting that's causing that multitude of writes. I wish that I understood the whys-and-wherefores of the source code to tell you just how you could achieve “manageable, clean source-code” while avoiding those to-you-useless and unwanted writes.

I frankly don't share the notion that you should use a tied-hash here, because if virtual-memory works for you it sure does have lots of benefits. But not when the VM manager of the operating system thinks it's volatile because you're whacking reference-counters... Those {how's the system to know they're...} unnecessary COW-faults are a serious recipe for thrashing, as no doubt you've seen.

Replies are listed 'Best First'.
Re^2: WHY copying does happen (fork)
by Anonymous Monk on May 09, 2008 at 15:52 UTC
    yeah, I would like to have this data structure and would be able to access it from all forks / threads. In theory threads would be nice, but sadly the way they are currently implemented in perl (shared data are not really shared, everything is damn slow) make them close to useless for us :(
    I am afraid I have to accept that these (for us uneccesary) write do happen and that we can not really avoid them. I am currently thinking of how we could rewrite the whole application to get the needed data from somewhere else (some suggested a in memory database) without taking a too large performance hit.