Re: die rather than exit on out-of-memory failure?

There is ... ahem ... a relatively simple solution to that “problem,” to wit:

Do not attempt to allocate a data-object of that size.

What you are actually doing, in such a situation, is “allocating a disk file the hard way.” All storage used by a process is, after all, &dquo;virtual storage,” and this means “disk file.” The virtual storage subsystem is not designed to handle a process hitting 200 megabytes’ worth of 4K pages all at once.

Such data should be stored in disk files. Those files can be processed in a variety of ways, such as the tie mechanism, or by mapping portions of them into the virtual-storage space, but one must never completely forget the purely physical aspects of disk storage: transfer time, rotational latency, and seek time.

Replies are listed 'Best First'.
Re^2: die rather than exit on out-of-memory failure? by chm (Novice) on Jan 04, 2011 at 03:05 UTC
The problem is that perl exit()s rather than die()s which make recovery on failure a bit problematic. Reducing the memory footprint of the application is a work around but in the case of PDL where the perl SV data is actually an opaque object being processed by optimized C computational kernels, it is not "relatively simple" to implement I'm thinking that a possible approach would be to replace the implicit perl memory allocation with our own calls to the system malloc routine with the perl object reference now being controlled by magic or some such... I'm fine with a failure to allocate these large objects but would like the perl interpreter not to exit. The current pdl_malloc using perl SVs is definitely far outside the bounds of expected perl usages. However, one of the goals of PDL was to make exactly such memory and computational performance problems accessible from perl.	[reply]
Re^3: die rather than exit on out-of-memory failure? by karlglazebrook (Initiate) on Jan 04, 2011 at 03:30 UTC
Indeed. Rather than explode the spaceship, there should be a light which says 'please do not press that malloc button again'. It is relatively easy to generate such a large memory request in PDL even accidentally, by the nature of the language, And 200 Mb objects aren't much these days in the era of Gb memory computers.	[reply]
Re^2: die rather than exit on out-of-memory failure? by Anonymous Monk on Jan 04, 2011 at 03:02 UTC
There is ... ahem ... a relatively simple solution to that “problem,” to wit: That wasn't the question. The question was about using the emergency memory space to die instead of exit.	[reply]
Re^3: die rather than exit on out-of-memory failure? by locked_user sundialsvc4 (Abbot) on Jan 05, 2011 at 03:33 UTC
Ahhh... `:{` Sorry! Mea culpa!* (“Yum! This foot tastes good!”)