There's no arbitrary limit that Perl imposes on itself.
It basically has two types of constraints: whatever constraints the OS implies on it (ulimit, process size limit,
total memory available, etc), and a limit that depends on
the pointer size. With 32 bit pointers, you won't be able
to go over 2 or 4 Gb. With 64 bit pointers, you can go further.
Also note that if Perl dies with an "Out of Memory error" while using 3.28 Gb, it doesn't mean 3.28 Gb is the limit.
It means the limit (at that moment in time) is between
3.28 Gb and whatever the amount it was that was being claimed.
If my processes were dying with "out of memory" errors, and
they were using over 3 Gb of memory, I'd first look at the
program and see whether I could gain by doing a redesign
that looking at imposed limits.
Abigail | [reply] |
The message means that perl asked the OS for more memory and got a refusal. That basically only happens if you ran out of both physical memory and swap space (assuming swapping is enabled). The point (memory consumed) at which this happens depends not just on the data size/memory used by your process, but also by the memory in use by other processes in the system. So if the other processes--system and user--are using more memory today than yesterday, then your process will hit the limits of your system earlier.
Update: Tilly pointed out that the above information could be misleading to those coming along later. This reply is aimed specifically at the OP who is using a 64-bit OS and therefore unlikely to being affect by any address space limitations. Anyone using a 32-bit OS may encounter an os-dependant, 2 or 4 GB limit for a given (perl) process even though his RAM + swap totals more than 2/4 GB.
| [reply] |
Perl doesn't have an inherent limit on process size. I ran into something similar to your situation on Linux where I was hitting a limit. It turned out to be the OS, and not Perl. I suspect it is the OS in your case as well. | [reply] |
Perhaps you should take one step back and look at your algorithm again, may be there is another approach that will consume less memory without sacrificing much processing speed.
| [reply] |
The addressable data space by a single process under Solaris in 32 bit mode is approximately 3.7GB (yes I know you are running 64 bit Solaris, but my guess is that your perl binary is compiled in 32 bit mode).
Under linux 32 bit (at least the non-Enterprise versions) the limit is just under 2 GB for a single process because of the way linux maps memory. On RH Advanced Server one can use a special kernel variable (mapped_base) to increase the adderssable memory of the process to 2.7GB.
Michael
| [reply] |