kabeldag has asked for the wisdom of the Perl Monks concerning the following question:

Hello folks. I have a problem. I am writing a program using Tk. Now the Tk process is started with MainLoop which blocks. You obviously need another way of updating/writing/reading from/to the Tk objects. I am currently aware of a few ways of doing stuff like this:

1. Use the repeat/after toplevel Tk methods
2. Fork a new process
3. Use threads
4. IPC ??? (Never used pipes much...)
5. Manually update display fields using callbacks ( : ()
* NO THANKS, IT AINT EXACTLY DYNAMIC *
6. Use Threads::Shared; - But this is new to me and Im not sure you can share anything but simple scalars

Now, what I have tried to do is, be able to call Tk methods on Tk objects from the seperate worker_thread, ie; access the Tk objects from within the seperate worker thread, in this case (update_thread). I know you can share scalars, and perhaps array's, but but but what about a module ? I heard that Tk is unsafe with threads atm !?
use strict; use Tk; use Threads; my ($mw,$sent_recvd_listbox,$server_list_listbox); threads->new(\&update_thread)->detach; create_tk_window(); $mw->MainLoop; sub update_thread { print "update_thread called...\n"; while(1) { if(defined($server_list_listbox)) { # This never gets executed because the objects from # the tk module are out of scope ?????? $server_list_listbox->insert('end',"Some string data"); }else{ print "Couldn't insert any data cause this thread can't se +e the tk objects !?\n"; } sleep 1; } exit(); } sub create_tk_window { $mw=MainWindow->new ( -background=>'#dedede', -foreground=>'yellow', -title=>"FingerLick" ); $mw->geometry("802x618"); $mw->minsize(802,618); $mw->maxsize(802,618); $sent_recvd_listbox=$mw->Listbox ( -height=>1, -width=>60, -background=>'black', -foreground=>'yellow' )->pack(-side=>'bottom',-anchor=>'s',-pady=>2); $server_list_listbox=$mw->Scrolled ( "Listbox", -height=>20, -width=>60, -background=>'white', -foreground=>'black', -scrollbars=>'se', )->pack(); }
Running the script above ^^ for me on a windows 2000 machine with v5.8.3, the update_thread (obviously) will just continue printing "Couldn't insert any data cause this thread can't see the tk objects !?" until you interrupt it.

If you create the seperate thread (threads->new(\&update_thread)->detach;) after calling create_tk_window, it gives me the following error:
Free to wrong pool 1e46660 not 15d3f10 at C:/Perl/site/lib/Tk.pm line 228.
(oops, forgot to edit the logical drive out of the path ;)

I have also breifly attempted to share a globally scoped (ie; defined at the top of file) IO::Handle that I would write/read to/from both update_thread and the main process. Ie; instead of trying to call $server_list_listbox->insert('end',"blah"); from inside the while loop in the update_thread. But no results..lol.

I know I could use the Tk repeat method to timeout every n amount of time and run subroutines that collect data outputted from various seperate 'worker' threads, but it seems like a real pain in the side to do that. Im thinking I would have to use 'magic numbers' or predefined values, that both threads relate to specific global shared data. Ie; I would have
my $wt_display_code:shared; my $wt_data:shared; $mw->repeat(0.001,sub { display_worker_data() });
Then, in the display_worker_data() sub, check some shared scalar that relates to what data the worker_thread has to offer, and just as importantly, where it has to go (what Tk widget field).

Eg;
my $wt_display_code:shared; my $wt_data:shared; $mw->repeat(0.001,sub { display_worker_data() }); sub display_worker_data() { if($wt_display_code eq "SENT_RECVD") { my $data_for_display="Sent recvd: $wt_data"; $sent_recvd_listbox->insert('end',$data_for_display"); }elsif($wt_display_code eq "TIME_STATUS") { my $data_for_display="Time: $local_time"; $time_stats_listbox->insert('end',"$data_for_display"); }elsif(...) { ... }elsif(...) { ... } etc... }
BUT THAT IS A NIGHTMARE ^^ The number of possible value pairs are endless.

Is there a way to have the Tk module objects shared in both threads, you know, being able to access the Tk objects from a seperate process/thread ?

Maybe I should be using a simple pipe to communicate data between threads. Don't tell me to use fork. Because it gave me more greif than you can poke a stick at, thats why I am trying to use threads.

As much help as possible is most appreciated. Thanks folks - kabeldag (DU).

Janitored by Arunbear - added readmore tags, as per Monastery guidelines

Replies are listed 'Best First'.
Re: Sharing Tk-module objects in threads
by BrowserUk (Patriarch) on Nov 05, 2004 at 03:32 UTC

    Try something like this. Making the lists scroll and a proper way to terminate the worker thread(s) is something gor you to read up on.

    use strict; use Tk; use Threads; use Thread::Queue; sub create_tk_window { my $mw=MainWindow->new ( -background=>'#dedede', -foreground=>'yellow', -title=>"FingerLick" ); $mw->geometry("802x618"); $mw->minsize(802,618); $mw->maxsize(802,618); my $sent_recvd_listbox=$mw->Listbox ( -height=>10, -width=>60, -background=>'black', -foreground=>'yellow' )->pack(-side=>'bottom',-anchor=>'s',-pady=>2); my $server_list_listbox=$mw->Scrolled ( "Listbox", -height=>20, -width=>60, -background=>'white', -foreground=>'black', -scrollbars=>'se', )->pack(); return( $mw, $sent_recvd_listbox, $server_list_listbox ); } sub update_thread { my( $Qservers, $Qxmit ) = @_; while(1) { $Qservers->enqueue( "Server" . int rand 1000 ) if rand() < .1; $Qxmit->enqueue( "Sent: la la la " . int rand 1000 ) if rand() + < .1; $Qxmit->enqueue( "Recv: do be do be do " . int rand 1000 ) if +rand() < .1; select undef, undef, undef, 0.1; } } my $Qservers = new Thread::Queue; my $Qxmit = new Thread::Queue; my( $mw, $srl, $slb ) = create_tk_window(); sub updateScreen { $slb->insert( 'end', $Qservers->dequeue ) if $Qservers->pending; + $srl->insert( 'end', $Qxmit->dequeue ) if $Qxmit->pending; } threads->new( \&update_thread, $Qservers, $Qxmit )->detach; $mw->repeat( 100, \&updateScreen ); $mw->MainLoop;

    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "Think for yourself!" - Abigail
    "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
      BrowserUk, you bloody champ you !!
      Guess what, I found out a way aswell ! Using IO::Pipe.

      I figured out the following way to achieve my goal but you beat me to it.. L: hehe. And you know what, It's the repeat Tk method aswell o_(.
      PLUS, I am using fork again. When I said I wasn't going to. But your way is very nice.
      use IO::Pipe; use Tk; $pipe = new IO::Pipe; create_tk_window(); if($pid = fork()) { # Parent $pipe->reader(); $mw->MainLoop; }elsif(defined $pid) { # Child $pipe->writer(); while(1) { write_to_pipe(); } } sub write_to_pipe { $pipe->write("Who's a jebroni - Not me Not me"); } sub display_handle_data { my $pipe_data; $pipe->sysread($pipe_data,100024); $server_list_listbox->insert('end',"GOT $pipe_data"); $server_list_listbox->see('end'); } sub create_tk_window { $mw=MainWindow->new ( -background=>'#dedede', -foreground=>'yellow', -title=>"FingerLick - Only eye nows the gows wehn the wind blo +ws" ); $mw->geometry("802x618"); $mw->minsize(802,618); $mw->maxsize(802,618); $mw->repeat(1.001,sub { display_handle_data() }); $sent_recvd_listbox=$mw->Listbox ( -height=>1, -width=>60, -background=>'black', -foreground=>'yellow' )->pack(-side=>'bottom',-anchor=>'s',-pady=>2); $server_list_listbox=$mw->Scrolled ( "Listbox", -height=>20, -width=>60, -background=>'white', -foreground=>'black', -scrollbars=>'se', )->pack(); }
      The Threads::Queue method that you have suggested may be the winner I think.
      We shall see after some tests...etc..

      Thank you very much BrowserUk. Now things are moving forward a lil more...
      I wanted to finish this thing by this weekend, looks like I may now.

      Thanks alot mate : _)
Re: Sharing Tk-module objects in threads
by thospel (Hermit) on Nov 05, 2004 at 11:06 UTC
    While you got several answers to your question, I'd like to point out that you might be trying to use Tk in a way that's not very natural.

    You normally set up the way you want your application to look (initially) at application startup and then enter the mainloop, which is then *supposed* to block. Things then happen because of "external events", which you already declared and mapped to callbacks.

    If you are for example writing a filemanager, things happen because you press buttons or select files from a list and these activate their corresponding callback. If you are writing something like an IRC client, things happen because you get I/O events on the irc socket, or the user fills in a text box and presses enter. These then again trigger their corresponding callbacks. If you are writing a monitoring application that shows the state of something every X seconds, you normally set up a repeat that collects that information, displays it and goes to sleep again until the next time. Callbacks again.

    Only quite rarely do you need something that keeps running *all the time* and then for example periodically updates the display to show its progress. If your case is like that, a seperate process or thread might indeed be a very sane thing to do. But even for these the more normal solution is to chop up the work in small pieces and make a subroutine that every time it gets called does a piece of the work still pending. Then you arrange for this routine to be called as an idle callback (or as a repeat with timeout 0 if it's *very* high priority). No threading or forking needed if you do it this way, but the price you pay is that you have to "invert" your work so that it's driven by callbacks. For complex operations this is usually easiest done by writing the work in terms of a state machine. While it sounds a bit complex, in reality I tend to find it easier than managing a separate process/thread, especially since everything will happen "synchronous", so you usually don't have to worry about things like synchronization and locking

      I liked your comments, but say I have a long running sql query using DBI which blocks. Do you think you could demonstrate your technique using with a long running DBI query inside of Tk that would work on a Win32 machine? The main window will not refresh while DBI runs unless you separate that process somehow. I have a hack that works, but I am interested in seeing an example of what you are saying.
      Thanks so much!
      JamesNC
        Unfortunately the case you are describing there is indeed one that can't be solved in the normal callback style. That assumes you can split up the work in pieces that never block for a long time. A long running query invalidates that assumption. So for that case you must indeed use a separate process/thread or use an event driven database library (which means not directly using DBI. Even though there are plans for it, DBI is currently not event driven).

        Though even in the case of a separate process I'd normally set up a socket or pipe style connection between the GUI process and the DBI process and handle the socket/pipe events with I/O callbacks at the GUI side. The DBI side would probably be straightfoward code doing blocking calls.

Re: Sharing Tk-module objects in threads
by pg (Canon) on Nov 05, 2004 at 03:04 UTC

    In the document for threads::shared pragmas, it says:

    "bless is not supported on shared references. In the current version, bless will only bless the thread local reference and the blessing will not propagate to the other threads. This is expected to be implemented in a future version of Perl."
      Couldn't I solve the problem by using a global IO::Handle to communicate between threads, or use a named pipe ? Help me out here : _)

        If I switch those two lines as you mentioned:

        create_tk_window();#this was after the next line threads->new(\&update_thread)->detach;

        It appears to work to some extent, but after a while, yuo get errors. However i tried twice, and got different message each time. Once it said:

        update_thread called... Free to wrong pool 223f40 not 1bcf528 at C:/Perl/site/lib/Tk.pm line 4 +06. Attempt to free unreferenced scalar: SV 0x1e7643c at C:/Perl/site/lib/ +Tk.pm line 247.

        That was the same error you mentioned that you saw. But the second time, I saw a different message (but the program does not fail for a long time, I killed it at the end):

        Attempt to free non-existent shared string '_TK_RESULT_' at C:/Perl/si +te/lib/Tk. pm line 247. Attempt to free non-existent shared string '_TK_RESULT_' at C:/Perl/si +te/lib/Tk. pm line 247.
Re: Sharing Tk-module objects in threads
by zentara (Cardinal) on Nov 05, 2004 at 13:56 UTC
    I don't have a quick example, but my first idea on this is to first declare a bunch of shared variables before you create any Tk. Then create your separate Tk threads, and have them watch for changes to their respective shared variable. When that shared var changes, the thread will call it's own sub which will then do it's own method on it's own objects. You will not be able to use a method from one thread on an object in another. All you can do is send signals and data between them. But WATCH YOUR MEMORY. When you start putting Tk into multiple threads, the gain can sky-rocket if you don't do it exactly right. And as Slaven Reszic points out, "threads are not supported in Tk, and what works today, may break in a future release". So sticking with pipes and IPC is really your best bet.

    I'm not really a human, but I play one on earth. flash japh

      It doesn't make much sense to try and run Tk in multiple threads, you (generally) only have one screen and only want one interface. Besides which, it's totally unnecessary.

      Using one thread to maintain the user interface, and using one or more other threads do do your processing, and only communicating that information between them is such a natural split of responsibilities (shades of MVC). And, it's very simple to code.

      It is so much easier to write your processing code as a single, "normal", top-to-bottom flow, than trying to split all your algorithms into iddy-biddy chunks that can be processed in under 1/10th of a second so as not to render the UI unresponsive.

      All the fears of synchronisation problems using threads are completely unfounded. In fact, trying to coordinate and synchronise all those iddy-biddy bits of code through a state machine and global variables is infinitely harder, and vastly more fragile.

      Breaking up processing into 1/10th second chunks is a real suck-it-and-see process to get right. And just when you have, the range of the values being processed changes, or the calculations you perform take slightly longer, and all the careful balancing of callbacks goes out the window.

      Run your carefully balanced--UI responsiveness -v- maximal processing throughput--event-driven callback code on a processor that is slower than the development machines and the UI responsiveness drops.

      Run it on a faster machine and you fail to get full benefit from that faster processor because your now serviceing the UI many times more frequently than necessary. Coding your Events and callbacks to dynamically account for processor speed and processor load is total nightmare. Been there, done that. Never again!

      Trying do real work in between servicing User Interface Events is stupid. If you've ever worked in an office with a publicised telephone number and tried to get real work down between answering:

      • B-ring...b-ring: "My screen doesn't work?"

        "Have you switched it on?"

        Now, where's that bug?

      • B-ring...b-ring: "My keyboard doesn't work?"

        "Have you kicked the cable out?"

        Step. Step. Step. Damn! I should've step into that sub not over it. Start again.

      • B-ring...b-ring: "My computer keeps beeping?"

        "Is something resting on the keyboard?"

        Step. Step. B-ring...b-ring: Step. Oh f*** it!.

      • "YES!"

      you'll understand this.

      Have a one or two people dedicated to interfacing with the users, who field and filter every call, and then pass over the real problems--and concise, filtered information--to specialists.

      One interface thread; one or more specialist worker threads; with just the minimum of required information being passed between them through some simple, reliable mechanism like Thread::Queue (or pipes or sockets).

      Loosely coupled, simple and effective.

      It's the same "loose coupling" philosophy that is (should be) foremost when designing OO code. Or, keeping Perl and HTML seperate.

      I don't know who Slaven Reszic is--I can guess and I could find out--but provided people learn to only use Tk (or any other objects) from within a single process, there is no (logical) reason why Tk and iThreads shouldn't be used together.


      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "Think for yourself!" - Abigail
      "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
        Sorry, that should be Slaven Rezic, he's probably the #2 guy in Tk behind Nick Ing-Simmons, and the expert on comp.lang.perl.tk. He's also a monk here, I believe named esserte.

        I agree with you that threads can be used with Tk, but I've also seen threaded programs which seem to run fine, blow up for no reason every 100th run.


        I'm not really a human, but I play one on earth. flash japh
Re: Sharing Tk-module objects in threads
by zentara (Cardinal) on Nov 05, 2004 at 16:41 UTC
    This is how I would have done what you tried in your code. As you can see, I kept Tk out of the worker thread.YMMV
    #!/usr/bin/perl use strict; use Tk; use threads; use threads::shared; my $data_out:shared = 0; my $data_in:shared = 0; threads->new( \&update_thread )->detach; create_tk_window(); ######################################################### sub update_thread { print "update_thread called...\n"; while (1) { $data_in = 'thread-processing'.$data_out; sleep 1; } exit(); } sub create_tk_window { my $mw = MainWindow->new( -background => '#dedede', -foreground => 'yellow', -title => "FingerLick" ); $mw->geometry("802x618"); $mw->minsize( 802, 618 ); $mw->maxsize( 802, 618 ); my $sent_recvd_listbox = $mw->Scrolled('Listbox', -height => 20, -width => 60, -background => 'black', -foreground => 'yellow' )->pack( -side => 'bottom', -anchor => 's', -pady => 2 ); my $server_list_listbox = $mw->Scrolled( "Listbox", -height => 20, -width => 60, -background => 'white', -foreground => 'black', -scrollbars => 'se', )->pack(); $mw->repeat(1000, sub{ $data_out++; $server_list_listbox->insert( 'end', "Sent $data_out " ); $server_list_listbox->see('end'); $sent_recvd_listbox->insert( 'end', "Recived $data_in" ); $sent_recvd_listbox->see('end'); }); MainLoop; }

    I'm not really a human, but I play one on earth. flash japh