goibhniu has asked for the wisdom of the Perl Monks concerning the following question:

I'm still having problems with multi-threaded Win32::Console output. I embarassed myself by wasting BrowserUK's time with really bad code, but I now have two versions that I think work.

This one does everything I want (with both use strict AND use warnings), but all output is with simple print statements and scrolls up the screen.

Whereas this one uses Win32::Console to put all the status updates from the threads on the same line. After the first thread ends, all output stops.

Any continued help would be much appreciated.


I humbly seek wisdom.
  • Comment on Still having problems with multi-Threaded Win32::Console

Replies are listed 'Best First'.
Re: Still having problems with multi-Threaded Win32::Console
by BrowserUk (Patriarch) on Aug 16, 2007 at 01:03 UTC

    Add this line after the use Win32::Console; line in your code, and you will see the behaviour you are wanting.

    *Win32::Console::DESTROY = sub{ };

    To be really proper, you should also add

    use Win32::Console; ... my $saved = \&Win32::Console::DESTROY; *Win32::Console::DESTROY = sub{ }; my $CONSOLE = Win32::Console->new(STD_OUTPUT_HANDLE); END{ $saved->( $CONSOLE ); }

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Wow. I would never have thought to override a method like DESTROY in a standard module. I guess this has solved my problem but raises some questions. For my edification could somone address these?

      • Does it seem right that DESTROY is getting called at the end of the thread when $CONSOLE was instantiated in main?
      • In the interest of TIMTOWTDI, is there a way to $conref = &share(\$CONSOLE) or something and pass the ref to the thread to keep it open even though the thread ends?
      • Is this a thing that should be reported to the maintainer of WIN32::Console?
      • Where should I go for more self study on this? perlobj / perltoot?


      I humbly seek wisdom.

        Here are my answers, but I'm just a user so take them with a pinch handful of salt.

        • Does it seem right that DESTROY is getting called at the end of the thread when $CONSOLE was instantiated in main?

          Short answer. Do you want a solution to the problem, or to rewrite history?

          Some limitations of threads, in particular those to do with sharing object instances across threads, are so ingrained in the single-threaded history of Perl, that they will only truly be corrected by a ground-up re-write of Perl itself. But hey. Someone, many someones, are already doing that.

          As Joost is discovering in threads, refcounting, XS, DESTROY, there are potential solutions to some of the limitations embedded deep inside the guts of perl, but they will often create as many knock-on problems as the solve.

          When the originators of threads::shared disallowed the sharing of blessed refs, they probably had perceived the weight of the underwater body that would come into play if they exposed that tip.

          In general, don't try to share objects, perl's internals aren't upto managing them. There are 'proxy object' solutions around, but I have my reservations about them.

        • In the interest of TIMTOWTDI, is there a way to $conref = &share(\$CONSOLE) or something and pass the ref to the thread to keep it open even though the thread ends?

          Also in general, sharing perl object by 'cloning' through closure will not produce the effects you are hoping for.

          However, in the case of Win32::Console (and some other similar Win32 system objects), because the perl code is nothing more than a very lightweight Perl wrapper around the Win32 system object, and carries little or no internal state at the Perl level, it is safe to share the object this way.

          This is because the internal state is managed by the win32 system, and was designed from the ground up to be reentrant and therefore thread-safe. You'll find that it is perfectly safe to use a Win32::Console object from multiple threads--even on multi-cpu boxes--without having to apply any kind of user locking. This is because the entire console subsystem was designed for use within threaded applications and so all of the locking is taken care of internally to the system.

          The only exception to this lies entirely within the Perl wrapper and related directly to Perl's own resource management. Perl's memory management wasn't designed for use with threads and anomalies like this, when DESTROY() gets called, slipped the net when threading was tacked on.

          Hence why my override hack is effective in this instance. In particular, it relies upon the knowledge that whenever you ask for a handle to STD_OUTPUT_HANDLE, you are getting a handle to a single instance (a singleton) of the system internal data-structures. So even if you were creating and destroying thousands of threads that were each obtaining their own STD_OUTPUT_HANDLEs, deferring the cleanup on those handles is going to leak just the handle, not the complex and large data structures and buffers they represent.

        • Is this a thing that should be reported to the maintainer of WIN32::Console?

          At this stage, there is little or nothing that the author could do about it. Until the threads::shared::_refcnt() internal routine is exposed, verified and documented, the best he could do is what I have done. But, the console API is a general purpose API and it is entirely possible to create many, many concurrent console objects, and deferring their cleanup until program termination may not be what the application user needs.

          In other words, this responsibility has to fall to each application programmer as there in no generic solution available until Perl gives the module author the means to determine the difference between a per-thread destructor and the last-existing-reference destructor.

        • Where should I go for more self study on this? perlobj / perltoot?

          I don't think you do. Almost none of those modules, nor the perl books available, have had any kind of update (beyond the 'threads are evil and scary and not a part of the 'One True Way' cautions) to bring them into the brave new world of threading.

          Indeed, with the general response to the advent of threading being something akin to 'duck and cover', the body of knowledge that would constitute the basis for any such learning materiale is contained as much within the posts in this place--those by myself, zentara and a few notable others--as in any other place that I am aware of.

          My purpose in showing how threads can be used in Perl to easily tackle a bunch of problems that are otherwise hard or very hard, is not to say that: iThreads are perfect; nor the solution to every multi-tasking problem; nor that there aren't limitations and caveats. Nor am I on commission.

          Mostly, its a case of trying to break the cycle of nobody uses threads because nobody uses threads. Because if nobody uses them, then limitations like this one will never be exposed. There will be no body of knowledge and experience built up. And nothing to guide the developers of the threads in Perl 6 as to what they should be looking to achieve.

        As an aside. I recently, in the last 2 days, made a discovery that may allow for efficient and safe inter-thread object sharing without going the proxy route. It is based in my long held and stated belief that allowing shared lexical variables is the basic flaw in the iThreads architecture.

        Shared data is, almost by definition, process global data. As such, I believe that all shared variables should have been limited to being global variables. Access could be controlled lexically using our, but the underlying data is, and should be global variables. The efficiency savings that would arise by following this already existing segregation of data spaces could be significant. Especially for non-threaded code running on threaded builds.

        Suffice it to say, the relative newness of the discovery means I have yet to explore the idea in detail, so I'm keeping it under wraps until I can talk about it on the basis of some authority.

        Woe. Did I really type all that. And say all those things. I guess I did. Now to skip spell checking and hit create before I chicken out.

        Let the down-vote fest begin.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.