in reply to Re^5: Techniques On Saving Memory
in thread Techniques On Saving Memory

BrowserUk,
I know what closures are and how they work. I'm asking how you are going to fake a closure?

Hmmm. Classic example of how things that are clear in your mind don't make any sense to anyone else because you assume they know what you are thinking. By faking the closure, I mean get the same effect - causing the reference count to increase by one so when it goes out of scope it doesn't reach 0.

To avoid that, you would need to keep a copy of the unsmashed reference around. To do that, you need to put it somewhere:

This is the exact thing I am proposing you don't have to do. It is in scope at the time you are stringifying it. If the only thing (which is my understanding) that is keeping it from being GC'd when it goes out of scope is the reference count - just increase it. You use the stringified address to look it up later, you GC it by lowering the ref count.

If the idea isn't clear (forget workable for now) then let me know and I will try a non-commentary approach.

Cheers - L~R

P.S. I didn't indicate that XS wouldn't be required though it would be really cool if it wasn't.

Replies are listed 'Best First'.
Re^7: Techniques On Saving Memory
by BrowserUk (Patriarch) on Mar 09, 2005 at 23:43 UTC

    So, forgetting all the stuff about faking closures and stuff, your proposing to "just increase the refcount"?

    Assuming that you can do that, what happens if the thing you just increased the refcount of, was previously deliberatly weakened by the caller for inclusion into self-referential structure? Or after they've given it to you?

    How does it play if your 'array' is initialised by 

    tie my @a, 'Compact::Array';

@a = ( 'default' ) x 1_000_000;

    [download]

    The refcount on the SV pointing at that constant string 'default' is now 1000000. That's a million chances for it never getting cleaned up properly.

    I'm not saying that this isn't possible to do--it is--but these are all things I have had problems with.

    It gets even more complicated when you start thinking of the effect of storing references to shared objects and how that plays with threads and threads::shared.

    I've spent a fair amount of time looking at the code in threads.xs & shared.xs that has to play similar games with reference counts.

    The guys that wrote that code are a darn sight cleverer than I, and still those modules have probably been the source of more leakage than amost any other. That tells me that it's not trivial, even if my failure to make it work properly is down to my stupidity.

    In each case, there are a subset of situations in which any given solution will work fine, and for a given specific task, they work okay, but for a general purpose solution, you need to consider all these possibilities and more.

    Examine what is said, not who speaks.
    Silence betokens consent.
    Love the truth but pardon error.
[reply]
[d/l]
      BrowserUk,
      Ok, we are no longer talking past each other. Ok, so it isn't that simple. Perhaps a 80% solution is that simple though. You lay out the ground rules. It is the user's perogative to break them or play nice. I know it doesn't help you with your problem but I think it might with mine.

      Cheers - L~R

[reply]

        I just found one example of the sort of thing that scares me in threads: 

            /* The code below checks that anything living on
       the tmps stack and has been cloned (so it lives in the
       ptr_table) has a refcount higher than 0

       If the refcount is 0 it means that a something on the
       stack/context was holding a reference to it and
       since we init_stacks() in perl_clone that won't get
       cleaned and we will get a leaked scalar.
       The reason it was cloned was that it lived on the
       @_ stack.

       Example of this can be found in bugreport 15837
       where calls in the parameter list end up as a temp

       One could argue that this fix should be in perl_clone
    */
      

    while (tmps_ix > 0) {
      SV* sv = (SV*)ptr_table_fetch(PL_ptr_table,     tmps_tmp[tmps_ix
+]);
      tmps_ix--;
      if (sv && SvREFCNT(sv) == 0) {
           SvREFCNT_inc(sv);
           SvREFCNT_dec(sv);
      }
    }

        [download]

        I do not even begin to understand the reasoning or magic that lies behind the need to serially increment and then decrement the refcount of an SV if you've just tested it and it is currently set to 0?

        It's this kinda thing that has made me wary of playing with refcounts.

        Examine what is said, not who speaks.
        Silence betokens consent.
        Love the truth but pardon error.
[reply]
[d/l]

In Section Seekers of Perl Wisdom