Unexpected 'Out of Memory'

Abigail-II has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Unexpected 'Out of Memory' by Corion (Patriarch) on Jan 13, 2004 at 16:38 UTC
To shine at least a small ray of hope, it fails for me as well, so it dosen't seem to be your system. It segfaults at the prophesized position(Access violation to an address 0x000000015), so I guess some pointer gets thrashed. My Debug-fu is too weak to tackle this though :-) Update:It seems that it could have something to do with the regex engine and how it matches, as I could get the `map` to pass by re-anchoring the regex: `map {printf "%3d is a square\n" => $_ if (1 x $_) =~ /^$re$/} 1 .. + $MAX;` [download] Summary of my perl5 (revision 5 version 8 subversion 2) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef usethreads=undef use5005threads=undef useithreads=define usemultip +licity=def ine useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32 -D_CON +SOLE -DNO_ STRICT -DHAVE_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS +-DUSE_PERL IO -DPERL_MSVCRT_READFIX', optimize='-MD -DNDEBUG -O1', cppflags='-DWIN32' ccversion='', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64 +', lseeksi ze=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -release -libpath:"d:\ +perl\5.8.2 \lib\CORE" -machine:x86' libpth=C:\PROGRA~1\MICROS~3\VC98\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib + comdlg32 .lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uui +d.lib wsoc k32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt. +lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool +.lib comd lg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib + uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msv +crt.lib libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib gnulibc_version='undef' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release -l +ibpath:"d: \perl\5.8.2\lib\CORE" -machine:x86' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL +_IMPLICIT_ CONTEXT PERL_IMPLICIT_SYS Built under MSWin32 Compiled at Nov 6 2003 12:48:55 @INC: D:/perl/5.8.2/lib D:/perl/site/5.8.2/lib . [download] `perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web` [download]	[reply] [d/l] [select]
Re: Unexpected 'Out of Memory' by Elijah (Hermit) on Jan 13, 2004 at 16:00 UTC
It appears it is maybe a system limitation or error not a script error or a bug in Perl (Unless you use a different version other than 5.8). I ran your original script on my RedHat Linux box and my Windows XP Laptop (running activeperl) and it ran fine on both. [root@thegnuage perl-scripts]# perl -c abi abi syntax OK [root@thegnuage perl-scripts]# chmod a+x abi [root@thegnuage perl-scripts]# ./abi 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square 64 is a square 81 is a square 100 is a square 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square 64 is a square 81 is a square 100 is a square 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square 64 is a square 81 is a square 100 is a square [download] The following is all I could capture from my Windows command prompt because XP's command.exe sux and I have not hacked the registry yet. `64 is a square 81 is a square 100 is a square 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square 64 is a square 81 is a square 100 is a square 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square 64 is a square 81 is a square 100 is a square` [download] What amount of Ram have you got in your system? Are you running many resource intensive apps at the same time as the script (Like any java apps)? You could have a program giving you a memory leak and be unaware of it therefore you really are out of memory. Try a reboot and run the script again with minimal system apps running. If after all that if you still get the error I would try reinstalling your perl interpretor or just compiling the script with perl2exe and then run it. If you still have the error I would say your memory sticks are going bad or one or all your DIMM slots on your systemboard are going bad and therefore giving erroneous indications.	[reply] [d/l] [select]
Re: Unexpected 'Out of Memory' by Abigail-II (Bishop) on Jan 13, 2004 at 16:14 UTC
What amount of Ram have you got in your system? Half a gig of RAM and a quarter of gig of swap space. Are you running many resource intensive apps at the same time as the script (Like any java apps)? No, the system is mostly idle. No java running. Just Netscape (approx 66 Mb) and X (about 100 Mb), and a few xterms running bash. I considered system resources, but it doesn't add up. Where the program runs out of memory is repeatable; it only does with 'map', and it doesn't run out of memory if you increase $MAX. That doesn't seem to be consistent with running out of system resources. The system is Redhat 7.3, running perl 5.8.2. Ah, I see I have 5.8.0 and 5.8.1 installed as well. It runs out of memory with 5.8.1 as well, but not with 5.8.0. Now, that's interesting. Abigail	[reply]
Re: Re: Unexpected 'Out of Memory' by Elijah (Hermit) on Jan 13, 2004 at 17:02 UTC
Ahh yes very interesting. It could be a specific version bug. Let us know what else you find.	[reply]
Re: Unexpected 'Out of Memory' by antirice (Priest) on Jan 13, 2004 at 16:35 UTC
This is pretty interesting. It worked fine with 5.6.1 and 5.8.0 on freebsd. It caused 5.8.2 to segfault on my debian box where 5.8.0 and 5.6.1 ran fine. It also crashed AS 5.8.2 whereas AS 5.6.1 ran fine under windows. Seems to be a 5.8.2 problem. Also, increasing $MAX to 200 makes the problem go away with both. Update: On 5.8.2 under debian, it segfaults for $MAX between 66 and 125 as well as between 254 and 354 (stopped testing at 400). On AS 5.8.2 it crashes for $MAX between 65 and 125. It also crashes with 255 and above (stopped testing at 275 since whenever it crashes, it brings up that window which I have to click). If you'd like to test it yourself, use the following: `#!/usr/bin/perl use strict; use warnings; my $MAX = shift; my $re = qr /^(1+)(??{"(?:$1){" . (length ($1) - 1) . "}" })$/; for (1 .. $MAX) {printf "%3d is a square\n" => $_ if (1 x $_) =~ /$re/} do {printf "%3d is a square\n" => $_ if (1 x $_) =~ /$re/} for 1 .. $ +MAX; map {printf "%3d is a square\n" => $_ if (1 x $_) =~ /$re/} 1 .. $ +MAX;` [download] Windows: `perl -le "print $_,system(qq{perl script.pl $_ > dump.out})?' NOK':' o +k' for (1..275)"` [download] nix: `perl -le'print $_,$",system("perl script.pl $_ > /dev/null")?"NOK":"ok +" for (1..275)'` [download] Update 2*: I've compiled 5.6.2 and 5.9.0 and tested it on both. Surprisingly, it fails for $MAX set to 125. In fact, every version of perl I tested on every platform I tested failed when $MAX was set to 125. Odd. antirice The first rule of Perl club is - use Perl The ith rule of Perl club is - follow rule i - 1 for i > 1	[reply] [d/l] [select]
Re: Unexpected 'Out of Memory' by BrowserUk (Patriarch) on Jan 13, 2004 at 17:04 UTC
It looks to me like you found a bug in 5.8.2 (or later?). I get a segfault at the same point that you get the "out of memory" error. P:\test>t 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square 64 is a square 81 is a square 100 is a square 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square 64 is a square 81 is a square 100 is a square 1 is a square 4 is a square 9 is a square 16 is a square 25 is a square 36 is a square 49 is a square # segfault occurs here. AppName: perl.exe AppVer: 5.8.2.808 ModName: perl58.dll ModVer: 5.8.2.808 Offset: 0006676a P:\test>perl -V Summary of my perl5 (revision 5 version 8 subversion 2) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef usethreads=undef use5005threads=undef useithreads=define usemultip +licity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef [SNIP] [download] Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "Think for yourself!" - Abigail Timing (and a little luck) are everything!	[reply] [d/l]
Re: Unexpected 'Out of Memory' by Abigail-II (Bishop) on Jan 13, 2004 at 17:10 UTC
Well, I installed valgrind on this box, and it came with a long list of reports, ending with: `==1340== More than 30000 total errors detected. I'm not reporting any + more. ==1340== Final error counts will be inaccurate. Go fix your program! ==1340== Rerun with --error-limit=no to disable this cutoff. Note ==1340== that errors may occur in your program without prior warning f +rom ==1340== Valgrind, because errors are no longer being displayed.` [download] I'll write a bugreport tonight. Abigail	[reply] [d/l]
Re: Re: Unexpected 'Out of Memory' by BrowserUk (Patriarch) on Jan 13, 2004 at 17:31 UTC
If it helps, the segfault on my system occurs at this instruction (marked '>') in the execution of Perl_mark_stack_grow, called from Perl_sv_clear. `Disassembly of Function perl58.dll!Perl_sv_clear (0x28066647) 0x2806674E: CALL 0x2805FB9D ; SYM:Perl_m +arkstack_grow 0x28066753: POP ECX 0x28066754: MOV ECX,DWORD PTR [ESI+0x40] ; <==0x28066 +74B(*-0x9) 0x28066757: MOV EAX,EBX 0x28066759: SUB EAX,DWORD PTR [ESI+0xC] 0x2806675C: PUSH 0x96 0x28066764: ADD EBX,0x4 0x28066767: SAR EAX,0x2 > 0x2806676A: MOV DWORD PTR [ECX],EAX 0x2806676C: MOV EAX,DWORD PTR [EBP+0x8] ; ARG:0x8 0x2806676F: MOV DWORD PTR [EBX],EAX` [download] Up to that point the stack is showing PID: 604 TID: 468 - Stack Contents for 0x2806676A 0x2806676A: perl58.dll:Perl_sv_clear + 0x0123 0x28066D47: perl58.dll:Perl_sv_free + 0x0108 0x28065787: perl58.dll:Perl_sv_force_normal_flags + 0x00AE 0x28060E74: perl58.dll:Perl_leave_scope + 0x0320 0x2805FB9A: perl58.dll:Perl_pop_scope + 0x001C 0x2803F50E: perl58.dll:Perl_sv_compile_2op + 0x5E1A 0x2805F97F: perl58.dll:Perl_runops_standard + 0x000C 0x2808A125: perl58.dll:RunPerl + 0x0086 0x00401012: perl.exe+0x00001012 0x77E814C7: GetCurrentDirectoryW + 0x0044 [download] which may or may not be useful in tracking down the problem? Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "Think for yourself!" - Abigail Timing (and a little luck) are everything!	[reply] [d/l] [select]
Re: Unexpected 'Out of Memory' by waswas-fng (Curate) on Jan 13, 2004 at 16:00 UTC
No idea works fine for me. [waswas-osx:/tmp] waswas% perl -V Summary of my perl5 (revision 5.0 version 8 subversion 1 RC3) configur +ation: Platform: osname=darwin, osvers=7.0, archname=darwin-thread-multi-2level uname='darwin hampsten 7.0 darwin kernel version 6.0: fri jul 25 1 +6:58:41 pdt 2003; root:xnu-344.frankd.rootsxnu-344.frankd~objrelease_ +ppc power macintosh powerpc ' config_args='-ds -e -Dprefix=/usr -Dccflags=-g -pipe -Dldflags=- +Dman3ext=3pm -Duseithreads -Duseshrplib' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemulti +plicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-g -pipe -pipe -fno-common -DPERL_DARWIN -no-cp +p-precomp -fno-strict-aliasing -I/usr/local/include', optimize='-Os', cppflags='-no-cpp-precomp -g -pipe -pipe -fno-common -DPERL_DARWIN + -no-cpp-precomp -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='3.3 20030304 (Apple Computer, Inc. build + 1495)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', + lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags ='-L/usr/local/lib' libpth=/usr/local/lib /usr/lib libs=-ldbm -ldl -lm -lc perllibs=-ldl -lm -lc libc=/usr/lib/libc.dylib, so=dylib, useshrplib=true, libperl=libpe +rl.dylib gnulibc_version='' Dynamic Linking: dlsrc=dl_dyld.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-bundle -undefined dynamic_lookup -L/us +r/local/lib' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL +_IMPLICIT_CONTEXT Locally applied patches: RC3 Built under darwin Compiled at Sep 12 2003 19:50:49 %ENV: PERL5LIB="/sw/lib/perl5" @INC: /sw/lib/perl5 /System/Library/Perl/5.8.1/darwin-thread-multi-2level /System/Library/Perl/5.8.1 /Library/Perl/5.8.1/darwin-thread-multi-2level /Library/Perl/5.8.1 /Library/Perl /Network/Library/Perl/5.8.1/darwin-thread-multi-2level /Network/Library/Perl/5.8.1 /Network/Library/Perl . [download] -Waswas	[reply] [d/l]
Re: Unexpected 'Out of Memory' by rdfield (Priest) on Jan 13, 2004 at 16:05 UTC
Works fine on WinXP running Activestate build 806 (5.8.0) rdfield	[reply]
Re: Unexpected 'Out of Memory' by markov (Scribe) on Jan 13, 2004 at 16:26 UTC
Next to the physical system limits, you may also bump into the user limits. It may be the cause, maybe not. See `ulimit -a`	[reply] [d/l]
Re: Unexpected 'Out of Memory' by Abigail-II (Bishop) on Jan 13, 2004 at 16:34 UTC
Except for the fact that my ulimit is unlimited, that doesn't explain why there's no Out of Memory if $MAX is increased. In fact, the program shouldn't even use any large amount of memory - created strings are lexical, and not longer than $MAX characters long. Abigail	[reply]
Re: Unexpected 'Out of Memory' by xenchu (Friar) on Jan 13, 2004 at 16:50 UTC
I tried your program on W2K using ActiveState Perl 5.8.2 build 808. When I ran it as written the program ran on until W2K killed it. When I took the map command out it worked perfectly. When I tried it with $MAX = 70; it again worked perfectly. But when I put the map line back in the program again ran on until W2K killed it. When I set $MAX = 200; it ran correctly, including the map line. It still ran fine without the map line. Putting the map line before the for block also ran correctly with 200 set. With $MAX = 100; W2K again stopped the program. HTH xenchu `The Needs of the World and my Talents run parallel to infinity.`	[reply] [d/l]
Re: Re: Unexpected 'Out of Memory' by Elijah (Hermit) on Jan 13, 2004 at 17:06 UTC
It appears it is a 5.8.1 and 5.8.2 version specific bug. Everyone who has ran it on 5.8.0 has had it run fine reguardless of platform OS.	[reply]
Re: Re: Re: Unexpected 'Out of Memory' by xenchu (Friar) on Jan 13, 2004 at 23:24 UTC
I am at a machine running RH Linux 9 with Perl 5.8.0 right now. I could run the program again here but the matter seems to be settled. If someone wants me to run it I will do so, otherwise I will let sleeping bugs lie. Thanks for writing, Elijah. xenchu Hey, I got *Fried* today! `The Needs of the World and my Talents run parallel to infinity.`	[reply] [d/l]