oxone has asked for the wisdom of the Perl Monks concerning the following question:

It seems that when running Activestate Perl (5.8.7) on Windows XP, a perl program which runs out of memory stops 'silently' without writing an "out of memory" message either to STDOUT or STDERR.

I also run Perl on FreeBSD which helpfully will state "out of memory" to tell you there was a problem, which is what I'd expect.

The behaviour on Windows is a concern because a process may appear to start and finish fine, when in fact it may have been cut short if the system runs out of memory.

You could replicate this by filling memory on purpose:

# Warning: this will consume all of your memory $hash{$count++}='xyz'x999 while (1);

So, my question is this: is there a way to somehow get Activestate Perl on Windows to report "out of memory" rather than stop silently?

Note that my problem isn't with memory management in code, but just the risk that if a process DID unexpectedly run out, it wouldn't tell me.

Replies are listed 'Best First'.
Re: No 'out of memory' message on Windows?
by stiller (Friar) on Feb 09, 2008 at 12:09 UTC
    Possibly a supervisor process should monitor your primary process to see if it's still making progress. You might touch somefile from within the memorymonster, and have the supervisor check that file not becoming too old.

    I've found

    if (my $chld = fork) { # supervise $chld from here } else { # unreliable process here }
    useful. hth

    Edit: Maybe you can make do with just confirming a successful run with a print "successfully done\n";

      Thanks! Not quite sure how I failed to think of the simple approach of printing "OK" or some such at the end, then checking for presence/absence of that, but that will do me fine in this instance. Pointing out the obvious is much appreciated!
Re: No 'out of memory' message on Windows?
by BrowserUk (Patriarch) on Feb 09, 2008 at 15:53 UTC

    Should anyone be interested, the error appears to be on line 108 of vmem.h (5.8.8 sources):

    Disassembly of Function perl58.dll!VMem::LinkBlock (0x280EED69) ; protected: ; #ifdef _USE_LINKED_LIST ; void LinkBlock(PMEMORY_BLOCK_HEADER ptr) ; { 0x280EED69: PUSH EBP 0x280EED6A: MOV EBP,ESP 0x280EED6C: PUSH ECX 0x280EED6D: PUSH ECX 0x280EED6E: MOV DWORD PTR [EBP-0x8],ECX ; VAR:this < +VMem *> ; vmem.h - Line 106 ; PMEMORY_BLOCK_HEADER next = m_Dummy.pNext; 0x280EED71: MOV EAX,DWORD PTR [EBP-0x8] ; VAR:this < +VMem *> 0x280EED74: MOV EAX,DWORD PTR [EAX+0x4] 0x280EED77: MOV DWORD PTR [EBP-0x4],EAX ; VAR:next < +_MemoryBlockHeader *> ; vmem.h - Line 107 ; m_Dummy.pNext = ptr; 0x280EED7A: MOV EAX,DWORD PTR [EBP-0x8] ; VAR:this < +VMem *> 0x280EED7D: MOV ECX,DWORD PTR [EBP+0x8] ; ARG:ptr <_ +MemoryBlockHeader *> 0x280EED80: MOV DWORD PTR [EAX+0x4],ECX ; vmem.h - Line 108 ; ptr->pPrev = &m_Dummy; 0x280EED83: MOV EAX,DWORD PTR [EBP-0x8] ; VAR:this < +VMem *> 0x280EED86: ADD EAX,0x4 0x280EED89: MOV ECX,DWORD PTR [EBP+0x8] ; ARG:ptr <_ +MemoryBlockHeader *> ### TRAP OCCURS HERE ###### TRAP OCCURS HERE ###### TRAP OCCURS HERE # +## + 0x280EED8C: MOV DWORD PTR [ECX+0x4],EAX ### TRAP OCCURS HERE ###### TRAP OCCURS HERE ###### TRAP OCCURS HERE # +## ; vmem.h - Line 109 ; ptr->pNext = next; 0x280EED8F: MOV EAX,DWORD PTR [EBP+0x8] ; ARG:ptr <_ +MemoryBlockHeader *> 0x280EED92: MOV ECX,DWORD PTR [EBP-0x4] ; VAR:next < +_MemoryBlockHeader *> 0x280EED95: MOV DWORD PTR [EAX],ECX ; vmem.h - Line 110 ; ptr->owner = this; 0x280EED97: MOV EAX,DWORD PTR [EBP+0x8] ; ARG:ptr <_ +MemoryBlockHeader *> 0x280EED9A: MOV ECX,DWORD PTR [EBP-0x8] ; VAR:this < +VMem *> 0x280EED9D: MOV DWORD PTR [EAX+0x8],ECX ; vmem.h - Line 111 ; next->pPrev = ptr; 0x280EEDA0: MOV EAX,DWORD PTR [EBP-0x4] ; VAR:next < +_MemoryBlockHeader *> 0x280EEDA3: MOV ECX,DWORD PTR [EBP+0x8] ; ARG:ptr <_ +MemoryBlockHeader *> 0x280EEDA6: MOV DWORD PTR [EAX+0x4],ECX ; vmem.h - Line 112 ; } 0x280EEDA9: LEAVE 0x280EEDAA: RET 0x4 ;******************************************************************* +*************

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      BrowserUk:

      So LinkBlock didn't verify the pointer was valid before trying to link it into the list?

      ...roboticus

        To be honest, I can't work out how it gets in there. The 100+ *alloc macros and functions in the Perl sources make it impossible to work out what is going on at the source level.

        And the use of C++ (style) classes for odd bits of the code, with runtime built dispatch tables throws my debugger's stack trace into loops. That means I can't follow it in that way. I can only tell you where it appears to be trapping.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: No 'out of memory' message on Windows?
by BrowserUk (Patriarch) on Feb 09, 2008 at 11:31 UTC
Re: No 'out of memory' message on Windows?
by roboticus (Chancellor) on Feb 09, 2008 at 13:53 UTC
    I tried it on cygwin and received:

    $ perl OOM.pl Segmentation fault (core dumped) $ cat OOM.pl #!/usr/bin/perl -w use strict; use warnings; my %hash; my $count; $hash{$count++} = 'xyz'x999 while (1);
    It took rather longer than I expected to fail.

    ...roboticus

Re: No 'out of memory' message on Windows?
by toma (Vicar) on Feb 10, 2008 at 07:55 UTC
    Recently I had a similar problem with perl5.8.8 on Cygwin. I think I was running out of memory in a print statement.

    It took me about three hours to figure out what the problem was. An error or warning would have helped. I am used to silly warnings about deep recursion. Running out of memory should rate at least a warning.

    I expect this problem to get worse as the limitations of 32 bit systems become increasingly confining.

    When I made a test program for exhausting memory, I got a helpful error message, "Out of Memory!" I have gotten the same helpful message when I ran out of memory due to a program bug. I think the problem of silent memory-induced process death is a problem only in certain scenarios.

    It should work perfectly the first time! - toma