That main sub is going to thrash the life out of the CPU (100% of one core (at least; possibly more) even when nothing is happening.)

How to fix that?

Moving to Tk instead of Win32::GUI to address the OPs problem is a mistake as well.

Well, it wasn't really meant to address the OPs problem, I just don't Win32::GUI much

The mistake the OP is making is to try handing the responsibility of adding the data to the GUI list back to the event-loop thread.

Which translates into what linechanges in OPs code?

:)

How to fix that?

Don't use yield:

threads->yield();
[download]

That effectively equates to sleep( 0 ), which whilst it ensures that other threads get a timeslice, equates to:

saveThreadContext;
restoreThreadContext;
[download]

when there are no other threads available to run; which is a cpu costly way to implement a delay.

Replace it with Win32::Sleep( 33 ) will ensure that the gui remains responsive at speed faster than the eye can detect, but will drop the cpu usage of that thread to near 0.

Which translates into what linechanges in OPs code?

Should be as simple as:

#######my $DataQueue = Thread::Queue->new();
...
#wait for data, print data to GUI textbox
#######while(Win32::GUI::DoEvents() != -1) {
#######    my $tmptxt = $DataQueue->dequeue();
#######    $main_text_window->Append($tmptxt);
#######}

Win32::GUI::Dialog();

sub mainloop {
    while (1) {
        #lots of page getting and number crunching goes here
##########        $DataQueue->enqueue("some data");
        $main_text_window->Append($tmptxt);
    }
}
[download]

Though that is untested.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

There's nowhere to add a DoEvents() call because the offending section of code is only one line (my $response = $mech->get( $url );) that takes a couple seconds to finish.

Using a thread to allow your GETs to run asynchronously from your GUI is a good idea.

Using a queue to ship the data back to the gui thread for adding to the listbox is both unnecessary (you can add it from the same thread as you get it); and totally flawed if you use the blocking dequeue() call to fetch the data to be added, because that call blocks.

You could use dequeue_nb(), but that creates its own set of problems.

I'd love to learn more about Win32::GUI, if you're up to it

I long ago gave up on Win32::GUI because the lack of documentation made every minor code addition or change a nightmare of searching for examples, and trial and error.

As for what to use as an alternative, I mostly do without. But were I to have a need for a gui that could not be avoided, I probably plump for using HTML5 and the browser.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.