in reply to Re^16: threading a perl script
in thread threading a perl script

Actually I am aware of occupying your precious time by explaining to me things

My time isn't particularly precious these days, and I'm happy to talk about threading. But like Stephen Fry, I do get rattled a little by someone who just keeps repeating the same thing over and over. ("Ceiling|"; "They're heavy!")

iThreads are "heavy" compared to kernel threads in low-level languages like C. This is undeniable. But that isn't a reason to either reject them or decry them. Trucks are heavy compared to cars, but that's because they need to be to perform their function. So it is with iThreads.

And so it is with integers in Perl. They are heavy compared to integers in low-level languages like C.

On my 64-system: In C, a million (native) integers occupies 8MB. In perl, those same million integers occupy 32MB.

But we forgive Perl that weight because of all the value-add that weight gives us. Numbers that transparently convert themselves from text to binary, integer to floating point, and back again as the program demands.

No mess of text to binary conversion routines: _tstof(), atof(), _wtof(), _tstoi(),  atoi(), _wtoi(), _tstoi64(), _atoi64(), _wtoi64(), _tstol(), atol(), _wtol(), _ttoi(), _wtoi(), _ttoi64(), _atoi64(), _wtoi64(), _ttol(), atol(), _wtol() and corresponding mess of binary to text conversion routines: _itoa(), _i64toa(), _ui64toa(), _itow(), _i64tow(), _ui64tow(), _ultoa(), _ultow(), _ultoa_s(), _ultow_s().

We recognise the costs of Perl's dynamic nature, and recoginise that they are outweighed by the benefits it brings to the simplicity of its programs and productivity of its programmers.

Similarly, the 'weight' of iThreads brings huge benefits.

As implemented, iThreads are not perfect by any stretch of the imagination. There are many ways in which that implementation could be improved. But for dynamic languages, the iThreads model is the only way to go, and will be much copied in the future.

Indeed, I believe that when the dust settles, compiled languages will provide threading that offers a similar model. Ie. Explicit-only shared state. The benefits of shared state concurrency, but only when you want it. It is almost common sense really. Sharing everything is just so dangerous.

Human beings are very bad at remembering what they've used and where. But compilers are single-minded and relentless at tracking such details. So, let the compiler yell at us and stop working if we start accessing the same data from multiple threads. Have it force us to mark shared data as such and so force us to think about the requirements and consequences of doing so. Then, anything not explicitly marked is safe, requires no locking, and runs at full speed.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^18: threading a perl script
by vkon (Curate) on Apr 28, 2011 at 08:55 UTC
    thanks

    after reading your messages I walked, slept and then thought again.

    Indeed, maybe scripting languages mostly share same difficulties, which prevent threads to be very light.
    It could be that many things are done non-optimal - because ithreads are late-adder into the language, but - given that they're working good - I suppose they are o-key.
    Maybe some optimizations could be added along the way: not cloning CODE (is it cloned now?) or maybe marking SVs that are designed not to be cloned?

    Contrast to that, it is known that in Perl6 some language construct are designed to be parallelizable into different CPU, and - all is fine, but - again - there were discussions on parrot-list that threading model is suboptimal and needs reorganization.
    Do you know anything about that?

    Ok, I will do more walking, sleeping and thinking about ithreads
    :)

    Here is my agenda:

    • Again, I am going to look into Tcl threads - how usable are they, what model etc
    • In Tcl, Tk is also not thread-safe, but...
    • if using Tcl/Tk from Perl, using different threads but connecting to Tcl/Tk in a single-threaded-way - is it possible that Tk GUI could be used from any thread, without allocating a special thread for GUI only?

    Another idea - given that you've stated that Liz was not right, and also considering that mentioned meditation article was long ago already - maybe its worth to write a disproof of her article?
    I think - if you compile this current thread, make some kind of review - this could be a great advantage?
    You've put number of efforts explaining things, maybe these efforts could be reused?

    Just a thought.

    Will you attend at nearest YAPC? (in Riga)

    Regards,
    Vadim.

      Maybe some optimizations could be added along the way: not cloning CODE (is it cloned now?)

      Yes. Code is cloned now. And almost certainly always will be.

      Perl5 was never designed for threading. Perl 5 byte code (opcodes or whatever you want to call then), were simply never designed to be reentrant, and it woudl be close to impossible to attempt to make them so now. But in reality, the duplication of code segments isn;t much of a problem, they aren't that big.

      That said, I think it would be both possible and desirable to provide the ability to create a bare thread. One that only clones the process global state and nothing else, leaving the programmer to load just what he needs in a particular thread.

      maybe marking SVs that are designed not to be cloned?

      I agree. IMO, the only non-shared, non-process-global data that should be cloned are closures. I see no reason to clone unreferenced, and therefore unreferencable pre-existing lexical data.

      It seems to me (without my fully understanding all the mechanisms and their implications), that this should be doable. Perl already has the ability to decide what are closures and capture them.

      IMO the simplest and most effective change that could be made would be to remove the tie magic on scalars embedded in aggregate structures. Instead of having to clone a reference to every scalar in a shared array, you would only clone the reference to the array, not each of its elements.

      I have some half working code that does this without requiring CORE changes. I might get it to work properly one day.

      there were discussions on parrot-list that threading model is suboptimal and needs reorganization. Do you know anything about that?

      No. I basically gave up on Parrot when I saw threading was essentially going to be left as an add-on. In needs to be built in from the ground up to work efficiently. I gave up again when I saw that the spec called for Parrot to support every kind of threading known to man and then some. Coooperative and preemptive; kernel and user; and event-driven and message passing and STM and Uncle Tom Cobbly an'all. Seems the easiest decision is no decision.

      tcl ... what model

      From what I read of your earlier tcl threading links, they appear to have gone a similar route to Perl--new interpreters in kernel threads. I didn't see anything about shared data; or whether code needs to be duplicated.

      if using Tcl/Tk from Perl, using different threads but connecting to Tcl/Tk in a single-threaded-way - is it possible that Tk GUI could be used from any thread, without allocating a special thread for GUI only?

      I know next to nothing about tcl and little about Tk. But I imagine the problems are very similar. Both were originally designed for strictly single-threaded operation so reentrancy was never a concern, never mind a priority. I think it would be equally difficult , essentially requiring a complete re-write to add reentrancy to them.

      maybe its worth to write a disproof of her article?

      I think not. It would be too little too late. Those that have an agenda against thrads will always choose to cite that reference no matter what refutations have been made.

      I think - if you compile this current thread, make some kind of review - this could be a great advantage? You've put number of efforts explaining things, maybe these efforts could be reused?

      I think that all the information is here for anyone interested to find it. Is there any advantage to moving it? And most of it has been said before and is there for those that look. I write best when it just flows from me. Once I start reviewing to any great extent, I get bogged down in the detail.

      And anything formal, would need to be authoritative to stand up to close scrutiny, and there are still areas of the internal implementation that I simply do not understand and have found no ready reference for gaining clarifications.

      If someone else wanted to try, I'd have no objections and would work with them to answer what questions I could.

      Will you attend at nearest YAPC? (in Riga)

      I do not attend YAPCs.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.