in reply to Re^14: threading a perl script
in thread threading a perl script

threads.xs is 5% simpler to initialize

Man. Are you incapable of rational thought? (Scan forward to 12.25 seconds and watch for 4 minutes.)

for sure, you already saw this famous article

Yes. And I know that Liz didn't know what she was talking about.

(Take a close look at the crap modules she littered the Thread::* namespace on cpan with if you doubt this. )

And neither do you. Your misunderstanding (and her's) is manifest.

Actually this unfortunate design has its historical explanations - there were 5005THREADS, which were developed in pre-5.6.0 perl and were designed to be lightweight. this stuck, fo rsome reasons, and 'ithreads' were added quickly as easier solution.

And that is the final nail of proof in your 'threading expertise' coffin. You simply haven't taken the time, (nor any time it seems), to attempt to understand the subject on which you are pontificating.

5005 threads were kicked into touch because it was impossible to make Perl's 'fat' data structures safe. Not just from user abuse, but even internally.

Ithreads are not an 'evolution of 5005 threads.

Do you know how this threading is done in Python and Ruby, BTW?

Yes. Both implement forms of 'green threads', which is to say, user space threading. IN other words, they run as a single OS thread, and implement a (crude) internal scheduler within that single OS thread.

Which means they rely upon a global interpreter locks. Which means they are very slow. Not just for access to shared state, but all state.

And they do not scale.

Because they use user space threading, to the OS they run under, they are a single threaded process. Which means they can only ever utilise one core at a time. No matter how many cores are available, or how many (pseudo)threads the program spawns, they are only ever going to execute one instruction at a time.

Bottom line, in the words of Python users Python threads suck!. Read the entire thread. Look for the acronym "GIL". Try and understand.

In the words of Ruby users, "Threads suck in Ruby. They are not native threads but are programatic threads in the ruby engine. If you have any thread that tries to open a socket, and the socket destination is not there, it will take a while for the socket to time out. During this wait, the whole ruby engine is frozen."

Perl 5 is (to my knowledge) the only dynamic, interpreted language that supports true, concurrent, scalable (kernel-based) threading, Imperfect for sure, but far less imperfect than other dynamic languages. And far simpler and more productive to use than any compiled languages.

This thread is more than deep enough. If you want to continue to display your ignorance of this subject, you can do so as a monologue.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^16: threading a perl script
by Corion (Patriarch) on Apr 26, 2011 at 07:53 UTC

    At least for Python, your assertion is partially incorrect. The GlobalInterpreterLock "prevents multiple native threads from executing Python bytecodes at once", but (other than that), Python threads are native OS threads. You don't get as much benefit from these threads while running inside of Python, but at least C extensions (and it seems also blocking system calls) can release the GIL to other code.

      You're right. It is much worse than that.

      It spawns native threads, but then uses them like grean threads. Except that the internal 'scheduler' is so poor that it has to run in perceptual debug mode. Thus totally preventing any form of usable concurrency.

      Its like saying: Of course every one can have the privilege of owning their own car, sit in their own private space, listening to their own choice of music. But to go anywhere, you have to wait for the car-train to arrive, join the queue and follow it wherever it goes until (if you are lucky) it goes past your exact destination. At which point you can leave.

      It has all the costs with none of the benefits.

      I guess it does serve to highlight how difficult it is to implement kernel threading in a dynamic language; and just how f***ing amazing iThreads really are. Imperfect for sure. But still amazing, given that they were retro-fitted without breaking backward compatibility and are so usable and scalable.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
Re^16: threading a perl script
by vkon (Curate) on Apr 26, 2011 at 09:24 UTC
    Actually I am aware of occupying your precious time by explaining to me things. (no sarcasm, actually)
    these are useful, thanks.

    I never said that 'ithreads' was evolution of 5005threads,

    I was surprised to read that And I know that Liz didn't know what she was talking about.

    Well, I learned something new and useful today,
    :) :)

    I never stated that I know much threads details, but I thought I understand some basics and drawbacks.
    o-kay, maybe my understanding needs to be adjusted, in a sence that threads are more usable than I initially thought.

    Now I wonder - are Tcl threads any better?
    I will look at http://www.tcl.tk/doc/howto/thread_model.html and http://www.tcl.tk/doc/howto/thread_model.html and will let you know.
    Actually some things in Tcl is done better - compared to Perl (starkits, GUI and virtual file systems), but - for me at least - Perl is much better to use.

    Ok, now I am away reading articles about python and ruby threading pointed by you,
    for which I especially grateful.

      Actually I am aware of occupying your precious time by explaining to me things

      My time isn't particularly precious these days, and I'm happy to talk about threading. But like Stephen Fry, I do get rattled a little by someone who just keeps repeating the same thing over and over. ("Ceiling|"; "They're heavy!")

      iThreads are "heavy" compared to kernel threads in low-level languages like C. This is undeniable. But that isn't a reason to either reject them or decry them. Trucks are heavy compared to cars, but that's because they need to be to perform their function. So it is with iThreads.

      And so it is with integers in Perl. They are heavy compared to integers in low-level languages like C.

      On my 64-system: In C, a million (native) integers occupies 8MB. In perl, those same million integers occupy 32MB.

      But we forgive Perl that weight because of all the value-add that weight gives us. Numbers that transparently convert themselves from text to binary, integer to floating point, and back again as the program demands.

      No mess of text to binary conversion routines: _tstof(), atof(), _wtof(), _tstoi(),  atoi(), _wtoi(), _tstoi64(), _atoi64(), _wtoi64(), _tstol(), atol(), _wtol(), _ttoi(), _wtoi(), _ttoi64(), _atoi64(), _wtoi64(), _ttol(), atol(), _wtol() and corresponding mess of binary to text conversion routines: _itoa(), _i64toa(), _ui64toa(), _itow(), _i64tow(), _ui64tow(), _ultoa(), _ultow(), _ultoa_s(), _ultow_s().

      We recognise the costs of Perl's dynamic nature, and recoginise that they are outweighed by the benefits it brings to the simplicity of its programs and productivity of its programmers.

      Similarly, the 'weight' of iThreads brings huge benefits.

      • Sheer simplicity.

        For many, many uses, a single keyword: async() is all you need to get access to true, scalable concurrency

      • Safety.

        Need to do two things concurrently? Throw one of them into an async{ ... }; block and you've got it. Done and dusted.

        No fears about accidental sharing. (Mostly) no need to lock anything. No deadlocks, livelocks or priority inversions to think about. It just works,

      • Usability.

        For the common, simple cases, adding concurrency to your single threaded code is almost trivial.

      • Perl has them.

        Do not under estimate how unique a dynamic language with usable, scalable, workable threading is.

      As implemented, iThreads are not perfect by any stretch of the imagination. There are many ways in which that implementation could be improved. But for dynamic languages, the iThreads model is the only way to go, and will be much copied in the future.

      Indeed, I believe that when the dust settles, compiled languages will provide threading that offers a similar model. Ie. Explicit-only shared state. The benefits of shared state concurrency, but only when you want it. It is almost common sense really. Sharing everything is just so dangerous.

      Human beings are very bad at remembering what they've used and where. But compilers are single-minded and relentless at tracking such details. So, let the compiler yell at us and stop working if we start accessing the same data from multiple threads. Have it force us to mark shared data as such and so force us to think about the requirements and consequences of doing so. Then, anything not explicitly marked is safe, requires no locking, and runs at full speed.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        thanks

        after reading your messages I walked, slept and then thought again.

        Indeed, maybe scripting languages mostly share same difficulties, which prevent threads to be very light.
        It could be that many things are done non-optimal - because ithreads are late-adder into the language, but - given that they're working good - I suppose they are o-key.
        Maybe some optimizations could be added along the way: not cloning CODE (is it cloned now?) or maybe marking SVs that are designed not to be cloned?

        Contrast to that, it is known that in Perl6 some language construct are designed to be parallelizable into different CPU, and - all is fine, but - again - there were discussions on parrot-list that threading model is suboptimal and needs reorganization.
        Do you know anything about that?

        Ok, I will do more walking, sleeping and thinking about ithreads
        :)

        Here is my agenda:

        • Again, I am going to look into Tcl threads - how usable are they, what model etc
        • In Tcl, Tk is also not thread-safe, but...
        • if using Tcl/Tk from Perl, using different threads but connecting to Tcl/Tk in a single-threaded-way - is it possible that Tk GUI could be used from any thread, without allocating a special thread for GUI only?

        Another idea - given that you've stated that Liz was not right, and also considering that mentioned meditation article was long ago already - maybe its worth to write a disproof of her article?
        I think - if you compile this current thread, make some kind of review - this could be a great advantage?
        You've put number of efforts explaining things, maybe these efforts could be reused?

        Just a thought.

        Will you attend at nearest YAPC? (in Riga)

        Regards,
        Vadim.