in reply to Re^2: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off
in thread XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off

It's even possible this might be a documentation/application problem: the PDL object is getting destroyed, and I argue that anything other than the new behaviour only worked by coincidence. But the timing (and thread "location") of that destruction is in the hands of the wider application. Obviously, I'll be happy to add any needed Perl-threads support, protected by #ifdef, if someone can tell me what that would be :-)

Are there any easy fixes for this, e.g. in the scripts being discussed here? Or does PDL::Parallel::threads need updating?

  • Comment on Re^3: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off
  • Download Code

Replies are listed 'Best First'.
Re^4: XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off
by marioroy (Prior) on Feb 28, 2022 at 23:18 UTC

    Hi, etj

    There is awesomeness with PDL 2.076 in better utilization of the L1/L2/L3 cache. That is something to be proud of and a moment of celebration. Unfortunately, 2.076 introduces an anomaly. IMHO try storing the $$.$tid somewhere. For PDL to be thread-safety friendly, it is important for cleanup to occur by the originating thread which instantiated the object.

    I will update the MCE demonstrations with a user_begin block to work around the issue. This I can do safely, because the piddles are instantiated by the main thread; i.e. $$.tid == $$.0

    Update: Looking at the code, MCE workers also form piddles for the Strassen demonstrations. So I cannot do this unless I save/check the thread ID.

    user_begin => sub {
    # PDL 2.076 introduced a regression causing all threads to perform
    # piddle destruction. PDL cleanup should occur once, preferably by
    # the thread which instantiated the object.
    no warnings;
    sub PDL::DESTROY { 0; }
},

    [download]

    Not able to do the above, I want to mention that the strassen_07_t.pl example works fine as is in the repo by removing the line introduced in PDL 2.076.

    SvOBJECT_off((SV *)it->sv);

    [download]
[reply]
[d/l]
[select]

      I searched the web and came across this page where the developer tries SvOBJECT_off. Stas states, "So we get all kind of problems when automatically dereferencing it."

      SvOK_off(sv);
SvIVX(sv) = 0;
SvOBJECT_off(sv);

      [download]

      It is warming to the heart (because of the frustrations at times folks hoping and/or making things threads-safe) to read, "A working solution is needed to make mp2 API perl-ithreads-safe as it's not at the moment, ...".

      Stas settled with the following instead.

      SV *sv = SvRV(obj);
if (sv) {
    /* detach from the C struct and invalidate */
    mg_free(sv); /* remove any magic */
    SvFLAGS(sv) = 0;  /* invalidate the sv */
}

      [download]

      I tried replacing SvOBJECT_off with mg_free in PDL 2.076.

      /* Clear the sv field so that there will be no dangling ptrs */
if (it->sv) {
    // SvOBJECT_off((SV *)it->sv);  /* problematic, issue #385 */
    mg_free(it->sv);  /* remove any magic instead */
    sv_setiv(it->sv,0x4242);
    it->sv = NULL;
}

      [download]

      etj, will that work? The Strassen demonstrations work fine and see no adverse effects during global destruction.

[reply]
[d/l]
[select]
        I have implemented this on the current git master branch, and intend to release it very soon, after I have made a couple more tweaks to the demos system which has finally got overhauled. Thanks for the amazing research!
[reply]

In Section Seekers of Perl Wisdom