in reply to Re: Slow evolution of Perl = Perl is a closed Word (NQP, parrot concurrency == Oh dear.)
in thread Slow evolution of Perl = Perl is a closed Word

Are we reading the same document?

There is what is contained in the PPD; and there is what is contained in the source files.

For example: Here are a couple of comment blocks drawn from the latest Parrot sources:

Create a local copy of the PMC if necessary. (No copy is made if it is marked shared.) This includes workarounds for Parrot_clone() not doing the Right Thing with subroutines (specifically, code segments aren't preserved and it is difficult to do so as long as Parrot_clone() depends on freezing).
Fixup a PMC to be sharable. Right now, reassigns the vtable to one owned by some master interpreter, so the PMC can be safely reused after thread death. In the future the PMC returned might be different than the one passed, e.g., if we need to reallocate the PMC in a different interpreter.

How do you interpret those?

My interpretation is that

  1. Parrot generated code is not reentrant. It cannot be called from a multiple threads concurrently.
  2. Parrot data structures (PMCs) are allocated on a per thread (interpreter) basis, and have to be cloned (copied by freezing and thawing) in order to be shared.

That is, as best as I can divine, repeating all the mistakes of the Perl 5 implementation.

To understand the fundamental (and I do not use that word lightly here) importance of reentrancy to threading, read some wisdom from people who have done it. Pay particular attention to section 12.3.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."
  • Comment on Re^2: Slow evolution of Perl = Perl is a closed Word (NQP, parrot concurrency == Oh dear.)
  • Select or Download Code

Replies are listed 'Best First'.
Re^3: Slow evolution of Perl = Perl is a closed Word (NQP, parrot concurrency == Oh dear.)
by Corion (Patriarch) on Sep 11, 2007 at 07:29 UTC

    If you're looking for the problems of tacking on threads afterwards, you can also take a look at the Python Global Interpreter Lock (and the discussion around it). While I'm not a fan of Guido van Rossum's striving for speed for speeds sake, the discussion highlights the problems that you encounter when you don't plan for share-everything threading/lightweight threading/fibers/coroutines from the start.

    Of course, there remain the semantical problems of the global name space of Perl, especially what happens in a share-everything model with the following code:

    sub quote_string { qq{"\Q$_[0]\E"} }; sub send_to_client { my $client_socket = shift; async { print {$client_socket} quote_string($_) for @_; }; }; function send_strings_to_client { local *quote_string = sub { qq{'\Q$_[0]\E'} }; send_to_client(@_); };

    With a share-everything, it's not really clear how overwriting the globs should be handled. I would be for "overwriting (code) glob entries is discouraged" and there should/could be a strict 'thread-globs' pragma that prevents you from assigning to glob entries. This should prevent 95% of all those problems.

      Corion. Sorry for taking so long to get back to you, but I was hoping for some ... um ... something. Anything. Some reaction from those "in the know", to my earlier posts. Something to indicate that I was wrong about Parrot. But nothing.

      A monk, so confident in his appraisal that I "had done no research", that he posted his "just cos your on a broken platform" condemnation, anonymously.

      Back in the other part of this thread, before erroneousBollocks decided to lift his misunderstanding to a new level, chromatic made a suggestion that he would listen if I had something useful to say. When I asked him off-line if he was seriously inviting me to make a proposal, he opened his reply with..."I'm not a fan of threading but...".

      And in that single phrase he summed up the entire problem. Nobody developing Parrot believes that threading is necessary. And while that situation continues, and whilst those with the insight to see beyond the 22 year old straight jacket that is POSIX, to a future (actually now. I just saw an ad for a 4-core machine for less than $1000), in which multiple processors are ubiquitous, and threading is the only way to usefully exploit them, keep quiet...perl is doomed.

      In just the same way as the Perl experience for Win32 users will never improve, while the key players, (according to one recent post here, from TimToady on down), continue to use a lame unix emulation (Strawberry Perl) as their only experience of Perl on Win32.

      Read again, and weep.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
Re^3: Slow evolution of Perl = Perl is a closed Word (NQP, parrot concurrency == Oh dear.)
by erroneousBollock (Curate) on Sep 11, 2007 at 03:34 UTC
    I really sincerely hope this doesn't come off as sarcastic, but...

    How do you interpret those?
    Well... depends on surrounding context. :-) I've not read the source code, but I'll play anyway.

    Create a local copy of the PMC if necessary. (No copy is made if it is marked shared.) This includes workarounds for Parrot_clone() not doing the Right Thing with subroutines (specifically, code segments aren't preserved and it is difficult to do so as long as Parrot_clone() depends on freezing).

    Well on first glance that comment seems to hark back to iThreads' "copy unshared data by default" nastiness... but on second reading (without better context) they're talking about PMCs that represent user code.

    Fixup a PMC to be sharable. Right now, reassigns the vtable to one owned by some master interpreter, so the PMC can be safely reused after thread death.
    I think it's clear that this comment pertains to the "shared everything" model rather than the "shared nothing model". It also sounds like exactly what I'd do for a "shared interpreter pool" scenario in a "shared everything" model.

    In the future the PMC returned might be different than the one passed, e.g., if we need to reallocate the PMC in a different interpreter.
    I definitely cannot devine any meaning from that one... what specifically about those words gives you pause?

    In short, I definitely wouldn't interpret those comments (together or separately) to mean as you've concluded. I'd need much more context (and naturally to read the damn code) before I made any such conclusion.

    Actually, if I must interpret those comments as a whole (a silly exercise at best), I get the following:

    • Shared PMCs are never copied.
    • In the "shared everything" model, code that touches shared PMCs needs to be cloned into the offending thread.
    • They're thinking already about what's safe (or not) when considering caching of PMCs in interpreter pools.

    You see how I did that? ;) I got a completely different meaning from the same comments.

    Seriously though, it's not like there's a CMM traceability document tying specific parts of the parrot implementation to words in the TDD... the parrot development has been highly incremental, striving for specific goals at each stage. I can only go by the discussion I read in the group's threads.

    Larry (below) has put my mind at ease about perl6 wrt concurrency, so now I guess I'll just pay a bit more attention to the relevant groups to see what's really going on wrt concurrency in parrot.

    -David

      In the "shared everything" model, code that touches shared PMCs needs to be cloned into the offending thread.

      Ask yourself why? Why does anything need to be cloned? There is only one reason. Fork emulation.

      But if you have a real fork, you don't need to do that.

      And if you don't have a real fork, the fork emulation benefits nobody, because you do not have, and cannot provide, all the other bits that go with it.

      Signals for instance. Sure, Parrot could attempt to emulate signals as perl 5 has done, but it will only ever work for other parrot processes. You won't get a SIGCHLD when a non-perl child terminates because the OS doesn't do that. And that's just the tip of the iceberg.

      Fork emulation just doesn't work, and perpectuating it serves no good purpose.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        In the "shared everything" model, code that touches shared PMCs needs to be cloned into the offending thread.
        Ask yourself why?
        Firstly, what I wrote was pure speculation because I had no context at all.

        I'll play devil's advocate once more.

        Why does anything need to be cloned. There is only one reason. Fork emulation.
        I can think of at least two reasons:
        • the PMCs in question could be continuations
        • cloning could be done on a limited scale to promote affinity
        In reality I just don't know exactly how they're planning to support whats written in the PDD. In a perfectly ideal "shared everything" model nothing needs to be cloned... but what I got from reading the PDD is that you still need to protect certain structures to keep the Task PMCs and scheduler info consistent and free from corruption in the event that a thread dies.

        There is only one reason. Fork emulation ... But if you have a real fork, you don't need to do that.
        It does indeed look like they plan to provide some sort of fork()-like facility to platforms without it. I don't read anything else into it than that.

        Fork emulation just doesn't work, and perpectuating it serves no good purpose.
        Parrot has to run on multiple platforms, if you're going to provide the unix fork() via HLL APIs at all, you need to provide something similar to all platforms. Certainly, there will be costs associated with that on particular platforms.

        In any case, for an HLL you can use threads directly instead. perl5 specifically exposed unix fork() semantics, perl6 doesn't have to expose fork(). In that scenario I imagine it'll still be available by importing a module that knows how to do it using parrot APIs.

        -David

        Update: fixed a couple of typos.