Re^4: A faster?, safer, user transparent, shared variable "locking" mechanism.

yes, I was relying on the simple user-space locks, assuming that you're sharing the data with other threads in the same program...

This is an excellent point. And one I've visited before.

I posed a similar solution to the Parrot team, that of using purely user space locking rather than OS/Kernel provided locking mechanisms for thread synchronisation, a couple of years ago. Back then, the idea was dismissed out of hand.

I accepted this at that time on the basis that Parrot has to run everywhere--including lots of exotic architectures (SMP, NUMA et al.), on many different processors, and under many different OSs; stuff which I simply wasn't conversant with or able to get up to speed on--so maybe there were places where user-space locking based around atomic instructions accessing shared, user-space memory simply wasn't possible.

Your bringing this up again has rekindled that spark of a question and I'm now wondering whether there are really any OS/Archtecture/cpu combinations that would be possible targets for Parrot and capable of running threads that don't provide the necessary primatives to support user-space-only locks? That's a difficult question to answer and even harder to prove.

There is also the rather simple expedient of hiding the locking behind macros (or prefereably functions, given my experience of trying to unravel Perl5 macros when things don't work). On those platforms where user-space locks are either not available or not easily providable, then the macros use OS provided mechanisms.

Thinking about this has resurrected another earlier notion that fell by the wayside as a result of having to defend the overall proposals I put forward. Simultaneously having to defend against the "Redmond just took a unix idea and renamed it" and "that's a Redmond-only solution" :(

The idea that you use a pool of mutexs keyed by the data (address) they are protecting, and reusing them cyclically. This was something I used years ago to improve the granularity of locking whilst avoiding the memory inflation of one mutex per shared data item. It worked well back then and was very tunable.

Hm. Despite the amount of thought I've already expended on this, I can see it occupying a lot more. Trouble is, most of it will probably be wasted time as nothing I say is likely to have much influence.

I'd be more than interested to hear the results of your further deliberations, even if it's only academic interest :)

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re^4: A faster?, safer, user transparent, shared variable "locking" mechanism.

Replies are listed 'Best First'.
Re^5: A faster?, safer, user transparent, shared variable "locking" mechanism. by roboticus (Chancellor) on Nov 04, 2006 at 03:33 UTC
I'd definitely persue user-space locking, for speed if nothing else. `/me wonders why the Parrotistas dismissed the idea....` I'm not terribly familiar with processors other than x86 and a few 8-bitters, but I've lurked on the linux kernel mailing list for a couple of years. I've not heard of any of the locking techniques in the kernel relying on being executed in a protected context--it seems that they normally rely on atomic operations. (Though there were a couple of interesting threads a few years back that talked about "memory barriers" ... perhaps that might be related. The memory barriers were for enforcing that prior R/W operations actually made it into memory before the next series of R/W operations.) My suspicion is that doing locking in a protected context is normally done merely to ensure that the actual mutexes/condition variables/etc. are in a protected area so an errant memcpy() can't lock everyone out of the system. WRT NUMA, etc., I would imagine that the cache coherency in the system would protect you from most multi-cpu interactions--as long as you keep your mutex in the same cache line as the value or handle of the variable you're protecting. I don't think it's wasted time to ruminate on these issues. It sharpens the brain for other tasks. And sometimes you come up with the right idea and make something better. If only I could find my ASM backups ... I could dig up a few routines I used in a robotics job and dust 'em off and play with 'em a bit. Ah, well... --roboticus	[reply] [d/l]

Replies are listed 'Best First'.

Re^5: A faster?, safer, user transparent, shared variable "locking" mechanism.
by roboticus (Chancellor) on Nov 04, 2006 at 03:33 UTC

/me wonders why the Parrotistas dismissed the idea....

I'm not terribly familiar with processors other than x86 and a few 8-bitters, but I've lurked on the linux kernel mailing list for a couple of years. I've not heard of any of the locking techniques in the kernel relying on being executed in a protected context--it seems that they normally rely on atomic operations. (Though there were a couple of interesting threads a few years back that talked about "memory barriers" ... perhaps that might be related. The memory barriers were for enforcing that prior R/W operations actually made it into memory before the next series of R/W operations.)

My suspicion is that doing locking in a protected context is normally done merely to ensure that the actual mutexes/condition variables/etc. are in a protected area so an errant memcpy() can't lock everyone out of the system.

WRT NUMA, etc., I would imagine that the cache coherency in the system would protect you from most multi-cpu interactions--as long as you keep your mutex in the same cache line as the value or handle of the variable you're protecting.

I don't think it's wasted time to ruminate on these issues. It sharpens the brain for other tasks. And sometimes you come up with the right idea and make something better. If only I could find my ASM backups ... I could dig up a few routines I used in a robotics job and dust 'em off and play with 'em a bit. Ah, well...

--roboticus

[reply]
[d/l]