Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Perl Tk crashes when mem usage exceeds 4G on Win10

by boleary (Scribe)
on Sep 23, 2022 at 10:48 UTC ( [id://11147080]=perlquestion: print w/replies, xml ) Need Help??

boleary has asked for the wisdom of the Perl Monks concerning the following question:

I have a Perl Tk GUI application that crashes after it exceeds 4GB of RAM usage and I have no issues exceeding 4GB when running tests in a console application (without Perl Tk)

Operating system: Microsoft Windows Version 10.0.19044.2006 Perl version: v5.30.3 Tk version: 804.036 (latest available on CPAN)

Perl spits out this error almost every time it crashes, but sometimes it crashes without an error:

Free to wrong pool 678ea0 not e228dd0 at .\common\GUI_TESTS\test_memor +y_hog_gui.pl line 41.

When searching for this error, everything I could find was multi-threading related, and our application does not use multi-threading. I thought it may be because we have something configured as 32-bit instead of 64-bit, so I followed the instructions in this question and found that everything is configured as 64-bit.

perl -V:ivsize # ivsize='8'; perl -V:ptrsize # ptrsize='8'; perl -V:archname # archname='MSWin32-x64-multi-thread';

Below is an example GUI application that crashes after the memory exceeds 4GB. I have boiled this down from our application and the crashing behavior is the same. The data structure that we use is obviously much larger, so I am cloning a simplified version of ours many times to pass the 4GB threshold.

Note that this sample application does not crash when running on a Centos7 VM (only on the Win10 host)

use strict; use warnings; use Tk; use Tk::LabFrame; use Clone; my $MAIN_WINDOW = MainWindow->new; $MAIN_WINDOW->minsize(400, 400); my @dataStructureClones = (); my $textBox; my $button_frame = $MAIN_WINDOW->LabFrame(-label => "Test", -relief => + 'groove', -borderwidth => 2)->pack(); $button_frame->Button( -text => 'Run Crashing Operation', -command => sub { my $dataStructureThatCrashes = { NETLIST_INFO => { EXTRA_PROPERTIES => { C_SIGNAL => {}, NET => {}, }, NET_LIST => [ # omitting this call will allow the program to exc +eed 4GB until after it finishes the loop { NL_INDEX => 0, } ] }, }; my $lastUpdate = time(); push @dataStructureClones, $dataStructureThatCrashes; for (1 .. 5000000) { if (time() - $lastUpdate > 1) { # omitting this call will allow the program to exceed +4GB $textBox->insert("end", "Cloning hash ($_)...\n"); $MAIN_WINDOW->update(); $lastUpdate = time(); } push @dataStructureClones, Clone::clone($dataStructureThat +Crashes); } } )->grid(-row => 0, -column => 0); $textBox = $MAIN_WINDOW->Scrolled( 'Text', -relief => 'groove', -background => 'light grey', -foreground => 'black', -wrap => 'char', -scrollbars => 'osoe', -width => 110, -height => 24, )->pack(-side => 'top', -fill => 'both', -expand => 1); MainLoop;

Replies are listed 'Best First'.
Re: Perl Tk crashes when mem usage exceeds 4G on Win10
by syphilis (Archbishop) on Sep 23, 2022 at 12:41 UTC
    When searching for this error, everything I could find was multi-threading related, and our application does not use multi-threading.

    Your app might not use multi-threading, but the perl you have was probably built with mutli-thread support.
    Such error messages have been reported on such perls in the past - and they're mostly not related to multi-thread applications.
    They're also often not related to Tk.

    To help conclusively identify your perl installation, could you tell us what perl -V outputs for you ?

    Cheers,
    Rob

      Hi Rob, Here is our perl -V output

      --edit is it interesting that the -cflags is -DWIN64 but the cppflags is -DWIN32

      C:\CadEnhance\workdir\NetBom\EndPointCompare>perl -V Summary of my perl5 (revision 5 version 30 subversion 3) configuration +: Platform: osname=MSWin32 osvers=10.0.19041.329 archname=MSWin32-x64-multi-thread uname='Win32 strawberry-perl 5.30.3.1 #1 Mon Jul 13 02:10:43 2020 +x64' config_args='undef' hint=recommended useposix=true d_sigaction=undef useithreads=define usemultiplicity=define use64bitint=define use64bitall=undef uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='gcc' ccflags =' -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -D__USE_MINGW_ANS +I_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLIC +IT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields' optimize='-s -O2' cppflags='-DWIN32' ccversion='' gccversion='8.3.0' gccosandvers='' intsize=4 longsize=4 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long long' ivsize=8 nvtype='double' nvsize=8 Off_t='long long' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='g++' ldflags ='-s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib"' libpth=C:\STRAWB~1\c\lib C:\STRAWB~1\c\x86_64-w64-mingw32\lib C:\S +TRAWB~1\c\lib\gcc\x86_64-w64-mingw32\8.3.0 libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 + -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 - +lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomd +lg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_ +32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 libc= so=dll useshrplib=true libperl=libperl530.a gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs dlext=xs.dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags='-mdll -s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c +\lib"' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES HAVE_INTERP_INTERN MULTIPLICITY PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under MSWin32 Compiled at Jul 13 2020 02:15:51 %ENV: PERL5DB="BEGIN { require "c:/GitHub/CE_TOOLS/PERL_DEBUGGER/bol_stw +bry_perl5db.pl"}" PERLDB_OPTS="PrintRet=0 DumpDepth=3 HistFile=./dbg_history.txt His +tSize=300 " @INC: C:/Strawberry/perl/site/lib/MSWin32-x64-multi-thread C:/Strawberry/perl/site/lib C:/Strawberry/perl/vendor/lib C:/Strawberry/perl/lib
Re: Perl Tk crashes when mem usage exceeds 4G on Win10
by kcott (Archbishop) on Sep 23, 2022 at 22:29 UTC

    G'day boleary,

    I ran a few tests and can confirm your findings.

    I made a very small change to your code to confirm completion:
    Was: if (time() - $lastUpdate > 1) {
    Now: if (time() - $lastUpdate > 1 or $_ == 5000000) {
    And added the shebang: #!/usr/bin/env perl
    Other than that, I ran exactly the code you posted
    (modified code: C:\Users\ken\tmp\pm_11147080_tk_mem.pl).

    I have Cygwin running on Win10:

    $ uname -a CYGWIN_NT-10.0-19044 titan 3.3.6-341.x86_64 2022-09-05 11:15 UTC x86_6 +4 Cygwin

    I have Perlbrew with a variety of Perl versions, each with Tk installed. I ran

    ken@titan /cygdrive/c/Users/ken/tmp $ ./pm_11147080_tk_mem.pl

    successfully to completion, without any error or warning messages, with these Perl & Tk combinations: 5.30.0/804.034, 5.32.0/804.035 & 5.36.0/804.036 (each using ~5.5GB memory, from visual inspection of Task Manager).

    I don't use the MSWin side of my machine very often, doing nearly all of my work with Cygwin; however, I did have an old Strawberry Perl version with Tk:

    Microsoft Windows [Version 10.0.19044.2075] (c) Microsoft Corporation. All rights reserved. C:\Users\ken\tmp>perl -v This is perl 5, version 26, subversion 3 (v5.26.3) built for MSWin32-x +64-multi-thread ... C:\Users\ken\tmp>perl -E "use Tk; say $Tk::VERSION" 804.034

    I ran the script three times:

    C:\Users\ken\tmp>perl pm_11147080_tk_mem.pl C:\Users\ken\tmp>perl pm_11147080_tk_mem.pl Free to wrong pool 25e3ed0 not 10e38400 at pm_11147080_tk_mem.pl line +43. C:\Users\ken\tmp>perl pm_11147080_tk_mem.pl Free to wrong pool 25c3ed0 not 2ee60830 at pm_11147080_tk_mem.pl line +43.

    Each run died at about 4GB memory usage (from visual inspection of Task Manager) with the Tk application completely shutting down. Like you, I either got "Free to wrong pool ..." or no output at all. With the added shebang and a blank line after it, my line 43 should be the same as your line 41:

    $MAIN_WINDOW->update();

    I had a look in "Active bugs for Tk" but nothing really looked like it was relevant to this. There's apparently 144 in the list; I could have missed something; a second pair of eyes wouldn't hurt.

    I would definitely advise raising a bug report. Before you do that, though, I'd await further reports from other monks. The more specific information you can supply, the easier it will be for maintainers to track down the problem. The type of such information could be, but is not limited to:

    • Occurs on MSWin only or others.
    • Occurs on Win10 only or others.
    • Occurs on Strawberry Perl only or others.
    • Occurs with Perl versions earlier than 5.26.3.
    • Occurs with Tk versions ealier than 804.034.

    I acknowledge that you may already know this. You can consider it as a useful checklist for others.

    — Ken

      Thanks for the detailed analysis Ken. I tried to reply last Friday, but forgot to send it and lost my writeup

      I'm pretty sure it happens only on Windows. I can try on Win11 as well but I would expect the same. I don't have my Win7 hosts anymore

      I really only have access to Strawberry Perl v5.30.x and the linux version I am running... I use perl2exe to distribute my code and I'm locked in on those versions

      I did take a look at the active bugs and the 11679 Tk not 64-bit clean was pretty interesting... I do think if that were the issue, it would crash on Linux hosts as well... The code the in tkGlue.c that the bug highlighted has been around for a very long time!

      thanks again for the feedback!

        I'd suggest waiting until 1st October before submitting the bug report. That'll give about a week (from the time of the original post) for any further feedback: PM tends to be quieter over the weekend, so there may be more to come.

        In your bug report, I'd stick to what you know: problem on these (list) Perl & Tk combinations; but no problem on these (other list) combinations. I would avoid conjecture altogether: "I'm pretty sure ..."; "I would expect ..."; and similar. It's likely that maintainers have their own testing environments: they can confirm what you report as well as testing other scenarios.

        I'd also include a link to this thread: your report can summarise findings; with the link back to here for details. It shows that you made an effort before raising the bug report. There may be additional information by the time someone gets around to looking at the problem.

        — Ken

      G'day Ken

      I just got back from a very nice 2 week visit to Melbourne, Australia. Not sure if you are Aussie or Kiwi? (Unfortunately, all I got to see of NZ was the inside of Auckland Airport)

      I've never raised an official bug report... (I did add an issue in the Spreadsheet Module Git Repo once, but that seemed to go nowhere)

      Are you be talking about submitting this to Perl.org, Strawberry Perl, or going directly to TK?

      Thanks for your help,

      Bill

        "I just got back from a very nice 2 week visit to Melbourne, Australia. Not sure if you are Aussie or Kiwi?"

        See the Location: in my home node. :-)

        "Are you be talking about submitting this to Perl.org, Strawberry Perl, or going directly to TK?"

        In the first instance, I'd raise this as a Tk bug. In "Active bugs for Tk", and similarly for other distributions, you find menus: "About rt.cpan.org" -> "How do I ...?" — which leads to "How to use rt.cpan.org" where you'll find information about bug reporting (amongst other things).

        — Ken

Re: Perl Tk crashes when mem usage exceeds 4G on Win10
by choroba (Cardinal) on Sep 23, 2022 at 15:30 UTC
    When I increased the 5000000 to 50_000_000, the program happily ate all my 32 GBs of memory and started swapping.

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

      on a win10 host??? what does your perl -V output look like?

        On a Linux box :-)

        map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11147080]
Approved by marto
Front-paged by kcott
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (5)
As of 2024-03-29 12:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found