Re: Out Of Memory error at 950MB with 14GB free RAM

950 Mb for a 50 Mb is a lot of memory, even for Perl. How are you storing your file? As an array of characters? Then the memory consumption is no surprise. But if you store the file as a single string, you should consume about 50 Mb of memory.

I'd first try to find out what is causing the memory consumption. It might not even be related to your input. Something of the form $a [[1]] = 1; could cause the out of memory problem as well.

Abigail

Comment on Re: Out Of Memory error at 950MB with 14GB free RAM Download Code

Replies are listed 'Best First'.
Re: Re: Out Of Memory error at 950MB with 14GB free RAM by didier (Vicar) on Feb 10, 2004 at 13:27 UTC
My God! Please, Abigail, could you explain why this expression is a problem? `$a [[1]] = 1;` [download] take close to 50 MB on my system. Many thanks.	[reply] [d/l]
Re: Out Of Memory error at 950MB with 14GB free RAM by Abigail-II (Bishop) on Feb 10, 2004 at 13:53 UTC
`[1]` is a reference. A reference in a numeric context gives you a memory address. Memory addresses are usually big. Storing an element out of range in an array will make Perl grow the array so it fits - creating undefined values to fill up the array. Storing an element using a big index will make Perl create a huge array, with all the elements taking two handfuls of bytes. I'd say you're lucky it takes only 50Mb on your system. It dies on one system I tried it on, and it used 420 Mb on another. Abigail	[reply] [d/l]
Re: (3) Out Of Memory error at 950MB with 14GB free RAM by ambrus (Abbot) on Feb 10, 2004 at 14:01 UTC
Try `perl -wle 'print 0+[1]';` it will print a very big number, as it converts the pointer to an anonymous list (`[1]`) to an integer. Thus, `$a[[1]]=1` creates a very large array, as it has to set the `(0+[1])`-th element of @a to 1. Update: Abigail-II was somewhat faster to submit an answer, and his is actually cleaner than mine.	[reply] [d/l] [select]