in reply to Re^7: Techniques On Saving Memory
in thread Techniques On Saving Memory

Devel::Pointer has a deref() function, which allows you to get you hands upon the address without literally stringifying it. What you loose, is the type and blessedness information. If you can get that all in a single call rather than messing around with parsing the stringified representation that would just make things simpler.

The other side is if it is possible to recover that information from the SV pointed at, rather than having to store it and related it back later when you unsmash it, that would make life a great deal easier and remove another level of storage requirement and lookup.

With both sides, plus the appropriate refcount manipulations, you have the possibility to retain a single 32-bit value as a handle to anything. That makes the whole compact array idea almost practical.

Examine what is said, not who speaks.
Silence betokens consent.
Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco.

Replies are listed 'Best First'.
Re^9: Techniques On Saving Memory
by japhy (Canon) on Mar 10, 2005 at 18:00 UTC
    You don't lose the type and blessed information, per se. The address, when turned back into a reference, regains that data, if you do it the way I did it in my XS code: RETVAL = newRV((SV *) addr); Doing it like that restores the reference fully.

    What I don't get, though, is what this is good for. The memory for the data structure must still be allocated. In which case, why not just use a reference (which is an SV) instead of a number stored in an SV?

    _____________________________________________________
    Jeff japhy Pinyan, P.L., P.M., P.O.D, X.S.: Perl, regex, and perl hacker
    How can we ever be the sold short or the cheated, we who for every service have long ago been overpaid? ~~ Meister Eckhart
[reply]
[d/l]

      Because you can concatenate a million 32-bit numbers into a single, (4 MB) SV, and avoid having to allocate the other 999,999 SVs which would require 12 MB minimum, and over 220 MB if they were all blessed references and you put them into an array. And much more if you put them into a hash for fast lookup.

      A packed string of equal sized elements has all the properties of an array. Very fast indexing and the ability to be spliced. And it uses a huge amount less memory.

      Examine what is said, not who speaks.
      Silence betokens consent.
      Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco.
[reply]
        Ok, I'm game. So, should I work on this as Devel::RecoverRef (or some other, more appropriate name), or should I contact the authors of the Devel::Pointer* modules?
        _____________________________________________________
        Jeff japhy Pinyan, P.L., P.M., P.O.D, X.S.: Perl, regex, and perl hacker
        How can we ever be the sold short or the cheated, we who for every service have long ago been overpaid? ~~ Meister Eckhart
[reply]
        This is exactly how an array is implemented. All you really want is to be able to set each element to be the alias of the appropriate SV in memory. Arrays, in memory, are just a blob of containing a list of pointers to SVs. You're just removing yourself from that by one in that instead of having your 4MB blob of pointers by known by perl as pointers, you treat it as a string, find the pointer you want, and get the pointer to be treated as such. There already modules out there for helping with the task you've just outlined and they'll solve it better than offloading it into user code.
[reply]

In Section Seekers of Perl Wisdom