chute has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

Having a bit of an issue I'm hoping someone can help me with. I've created a script that uses Win32::GUI. The script retrieves webpages full of numbers and stuff, parses the numbers, compares them, etc. Processing can take minutes or more.

While the retrieving and processing is being done, the gui is sluggish and unresponsive. To solve this I discovered threads and implemented them, but it did not fix the problem. Everything looks correct to me, so I'm not sure what I can do to fix it. I'm using Strawberry 5.16.3, if that matters.

Here is a (heavily condensed) version of the code:

use strict; use Win32::GUI(); use WWW::Mechanize; use HTTP::Cookies; use threads; use Thread::Queue; #define some variables my @somearray = ("stuff"); my $thr; my $DataQueue = Thread::Queue->new(); #do all of the create windows/add control stuff my $main = Win32::GUI::Window->new( -name => "main", ); my $main_text_window = $main->AddTextfield( -name => "main_text", -multiline => 1, -readonly => 1, -vscroll => 1, ); $main->Show(); #call the main loop in a new thread $thr = threads->create(\&mainloop, @somearray); #wait for data, print data to GUI textbox while(Win32::GUI::DoEvents() != -1) { my $tmptxt = $DataQueue->dequeue(); $main_text_window->Append($tmptxt); } sub mainloop { while (1) { #lots of page getting and number crunching goes here $DataQueue->enqueue("some data"); } }

Also, I know about placing DoEvents() in the mainloop, but it slows down the processing quite a bit and does not help during page retrieval. I've also tried Win32::GUI::Timer (putting the dequeue and append command in the _Timer sub). It still seems to freeze when pages are being retrieved.

thanks!

Replies are listed 'Best First'.
Re: Win32::GUI window freezing, even with threading.
by BrowserUk (Patriarch) on May 28, 2013 at 06:26 UTC

    What do you think happens here if there is no data in the queue?

    #wait for data, print data to GUI textbox while(Win32::GUI::DoEvents() != -1) { my $tmptxt = $DataQueue->dequeue(); $main_text_window->Append($tmptxt); }

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      I took that from http://perl-win32-gui.sourceforge.net/cgi-bin/docs.cgi?doc=reference-methods#dialog

      I think that will execute my code block as long as the GUI isn't doing something.

      I could be wrong though.
        I took that from http://perl-win32-gui.sourceforge.net/cgi-bin/docs.cgi?doc=reference-methods#dialog

        There is no mention of "dequeue" anywhere on that page.

        The problem is not the bit you did take from that page; it is the bit you added:

        #wait for data, print data to GUI textbox while(Win32::GUI::DoEvents() != -1) { my $tmptxt = $DataQueue->dequeue(); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ $main_text_window->Append($tmptxt); }

        If the queue is empty, the call to dequeue() will block and your GUI will freeze.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Win32::GUI window freezing, even with threading.
by Anonymous Monk on May 28, 2013 at 04:07 UTC
    Try like this Re: threads causing memory leak (No leak!)(Win32::GUI DoOneEvent threads poll) , maybe organized like this
    #!/usr/bin/perl -- use strict; use warnings; use threads; use Thread::Queue; ## how $guithread and $mechthread communicate my $DataQueue = Thread::Queue->new(); my $guithread = threads->create( sub { require MyApp::GUI; ## MyApp/GUI.pm MyApp::GUI::go(@_); ## uses timer to check on queue }, $DataQueue, ); my $mechthread = threads->create( sub { require MyApp::Mech; ## MyApp/Mech.pm MyApp::Mech::go(@_); }, $DataQueue, ); $guithread->join; ## wait for gui to finish
      That seems to work the exact same way... Which is confusing because both threads should function independently of each other.

        That seems to work the exact same way... Which is confusing because both threads should function independently of each other.

        What seems to work the exact same way (did you adopt that code to your use case)?

        Consider Re: Thread parallel execution (join blocks), and throw Thread::Queue in the mix, does dequeue block like join (what I think BrowserUk hints at)?

        Thread::Queue SYNOPSIS lists a Non-blocking dequeue

Re: Win32::GUI window freezing, even with threading. (win32-gui-tk-thread-dequeue.pl, Thread::Queue::popnow/push)
by Anonymous Monk on May 28, 2013 at 07:51 UTC

    win32-gui-tk-thread-dequeue.pl, Thread::Queue::popnow/push

    If you write your Win32::GUI like this (like I discuss here and BrowserUk demonstrates) it should work the same way

    #!/usr/bin/perl -- ## perltidy -olq -csc -csci=10 -cscl="sub : BEGIN END" -otr -opr -ce +-nibc -i=4 -pt=0 "-nsak=*" use strict; use warnings; use threads; use threads::shared; use Thread::Queue; Main( @ARGV ); exit( 0 ); sub Main { my $qin = Thread::Queue->new(); my $qout = Thread::Queue->new(); my $guithread = threads->create( \&tkgui, $qin, $qout ); ## don't wait for background downloading service / mechtitles threads->create( \&mechtitles, $qin, $qout ); $guithread->join; ## wait for gui to finish return; } ## end sub Main sub mechtitles { my( $qin, $qout ) = @_; threads->detach(); ## can't join me :) require WWW::Mechanize; my $ua = WWW::Mechanize->new( autocheck => 0 ); while( 1 ) { if( defined( my $url = $qin->popnow ) ) { $ua->get( $url ); my $title = eval { $ua->title }; $title ||= $ua->res->status_line; $qout->push( "$url =>\n $title\n" ); } threads->yield(); } } ## end sub mechtitles sub tkgui { my( $qin, $qout ) = @_; require Tk; require Tk::ROText; my $mw = Tk::tkinit(); my $pending = ""; my $l = $mw->Label( -textvariable => \$pending )->pack; my $t = $mw->ROText()->pack; my $b = $mw->Button( -text => 'enqueue another 3 example.com', )-> +pack; $b->configure( -command => [ \&q_pusher, $b, $qin, ], ); $b->focus; $mw->repeat( 500, ## ms [ \&pop_to_pending, $t, \$pending, $qin, $qout, ], ); $mw->MainLoop; return; } ## end sub tkgui sub q_pusher { my( $b, $qin ) = @_; $qin->push( 'http://example.com' ) for 1 .. 3; $b->configure( -state => "disabled" ); return; } sub pop_to_pending { my( $t, $pending, $qin, $qout ) = @_; if( defined( my $item = $qout->popnow ) ) { $t->insert( q!end!, join( '', $item ) ); } $$pending = 'Pending ' . $qin->pending; return; } sub Thread::Queue::push { goto &Thread::Queue::enqueue } sub Thread::Queue::popnow { goto &Thread::Queue::dequeue_nb }

      That main sub is going to thrash the life out of the CPU (100% of one core (at least; possibly more) even when nothing is happening.)

      Moving to Tk instead of Win32::GUI to address the OPs problem is a mistake as well.

      The one, single advantage of Win32::GUi over the generic GUI frameworks, is that it was designed from the ground up to work in a multi-threaded environment.

      The mistake the OP is making is to try handing the responsibility of adding the data to the GUI list back to the event-loop thread. Whilst this is require of Tk and other 'portable' GUI frameworks -- because they are not thread-safe -- it is *NOT* a requirement of Win32::GUI.

      He can simple do away with the queue and move the $main_text_window->Append($tmptxt); into the same thread as the data is being gathered and allow the fact that pretty much every Win32::GUI Window function or method is actually implemented internally in terms of SendMessage(). Ie. the data to be append is is queued to the GUIs event loop via the process's system message queue to be processed in a timely fashion.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      hr //i

        That main sub is going to thrash the life out of the CPU (100% of one core (at least; possibly more) even when nothing is happening.)

        How to fix that?

        Moving to Tk instead of Win32::GUI to address the OPs problem is a mistake as well.

        Well, it wasn't really meant to address the OPs problem, I just don't Win32::GUI much

        The mistake the OP is making is to try handing the responsibility of adding the data to the GUI list back to the event-loop thread.

        Which translates into what linechanges in OPs code?

        :)

        This is how I set it up initially (because I didn't even know about threading). The data is added to the message queue to be processed by Win32::Gui to be processed when it gets around to it. My problem is whenever there is a WWW::Mechanize get call the gui would freeze. There's nowhere to add a DoEvents() call because the offending section of code is only one line (my $response = $mech->get( $url );) that takes a couple seconds to finish.

        A solution was posted below, but I'd love to learn more about Win32::GUI, if you're up to it (documentation and discussion is a bit sparse).