XS module in ithreads Perl much slower in threads::join after adding SvOBJECT

etj has asked for the wisdom of the Perl Monks concerning the following question:

Dear XS-knowledgeable Monks,

In https://github.com/PDLPorters/pdl/issues/385 the mighty marioroy has identified that PDL 2.076 is notably slower than 2.075 when exiting after using Perl threads, and specifically that this was after adding:

       SvOBJECT_off((SV *)it->sv); /* it = the struct pdl * */
[download]

in the PDL C-level destroy code (so that the PDL::DESTROY method would not be called on that SV, as that SV's IV, which is a pdl* pointer, gets set to an invalid value and we don't like SEGV).

In particular, as shown in the discussion on that GitHub issue, it takes about 1.5s in threads::join. Anyone got any idea why, and what to do about it?

Comment on XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off Select or Download Code

Replies are listed 'Best First'.
Re: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by dave_the_m (Monsignor) on Feb 28, 2022 at 13:41 UTC
I haven't ploughed through that long thread, but here are some basics that might point you in the right direction. When a thread is created, the whole active interpreter at that point is cloned (copied) into the new thread. When a thread ends, whatever SVs it returns (either explicitly via return, or implicitly as the last statement, or for XS, whatever SVs it pushes into the stack) are cloned back into the parent thread, by the parent calling the join() method. If the return SVs are non-trivial (e.g. have been blessed into a package), then additional stuff needs to be cloned back. For example if the object is in class X and the 'X' package doesn't exist in the parent (say 'use X' was done in the child, not the parent), then the X package and all the methods in that namespace need copying back too. What you are seeing is probably a situation where a large 'reverse clone' is being triggered by one of the SVs being returned to the parent. One possibility related to objects is the SVphv_CLONEABLE flag, which if set on the stash the object has been blessed into, triggers reverse-cloning the stash of that object. One possible side effect of turning off SvOBJECT() on the object is that it causes the SVphv_CLONEABLE test to be skipped (that test is only done on objects) which somehow causes the the stash associated with the (now unblessed) object to be cloned. But that's just wild speculation at the moment. Dave.	[reply]
Re^2: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by etj (Priest) on Feb 28, 2022 at 20:50 UTC
Nearly all of the thread is about other performance stuff, and was only linked here for completeness/interest; the heart of it for this purpose is as stated. Thank you, the above is very plausible and I have opened an issue on PDL::Parallel::threads quoting your wisdom: https://github.com/run4flat/PDL-Parallel-threads/issues/6.	[reply]
Re: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by marioroy (Prior) on Feb 27, 2022 at 23:43 UTC
It just dawned on me. Shouldn't destroy (or any re-assignment) happen in the thread and/or process that the the object was created in? i.e. $$.$tid Basically, to not DESTROY the object unless the owner ($$.$tid) which instantiated the object.	[reply]
Re^2: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by Anonymous Monk on Feb 28, 2022 at 08:33 UTC
Yes, basically vars arent shared by default, they live/die in their own threads. hows the memory usage affected by SvOBJECT_off? sounds like before SvOBJECT_off was added, the memory was being freed during global destruction , thus it was faster as a side effect... ?? Why no strassen with plain threads module? all those my $a really niggle	[reply]
Re^3: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by etj (Priest) on Feb 28, 2022 at 21:18 UTC
It's even possible this might be a documentation/application problem: the PDL object is getting destroyed, and I argue that anything other than the new behaviour only worked by coincidence. But the timing (and thread "location") of that destruction is in the hands of the wider application. Obviously, I'll be happy to add any needed Perl-threads support, protected by `#ifdef`, if someone can tell me what that would be :-) Are there any easy fixes for this, e.g. in the scripts being discussed here? Or does PDL::Parallel::threads need updating?	[reply] [d/l]
Re^4: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by marioroy (Prior) on Feb 28, 2022 at 23:18 UTC
Re^5: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by marioroy (Prior) on Mar 01, 2022 at 08:39 UTC
Some notes below your chosen depth have not been shown here