in reply to Re^6: Tracking down an Lvalue bug?
in thread Tracking down an Lvalue bug?

The patches for RT#67838 are is found in:

5.13.4+ 5.14.0+

I ran your test program with ActivePerl 5.14.0 (32 bit) on Windows, and noticed NO memory increase. (Steady at 99MB.) In fact, I saw in increases only up to that version of Perl.

perl -E"$x=x; $x x= 100e6; <>; $r = \substr $x, 50e6; <>; ord $$r; <>" AP 5.12.4 32-bit 99M 148M 148M AP 5.14.0 32-bit 99M 99M 148M AP 5.14.2 32-bit 99M 99M 148M

There will be a memory increase if you read the string $$r, but that's how magic works in Perl.


I looked at the source code at the time of the patch. There is no intentional "prefetching", and testing shows no accidental prefetching:

>\progs\perl5142-ap1402\bin\perl -MDevel::Peek -E"$x=chr(0); $x x= 100 +; Dump substr $x, 50;" SV = PVLV(0x4de51c) at 0x497b04 REFCNT = 1 FLAGS = (TEMP,GMG,SMG) IV = 0 NV = 0 PV = 0 <----- No buffer MAGIC = 0x4cc694 MG_VIRTUAL = &PL_vtbl_substr MG_TYPE = PERL_MAGIC_substr(x) TYPE = x TARGOFF = 50 TARGLEN = 50 TARG = 0x4a932c SV = PV(0x25603c) at 0x4a932c REFCNT = 2 FLAGS = (POK,pPOK) PV = 0x4b36cc "\0\0\0...\0\0\0"\0 CUR = 100 LEN = 104

In contrast with a version that does grow as soon as substr is called:

>\progs\perl5124-ap1205\bin\perl -MDevel::Peek -E"$x=chr(0); $x x= 100 +; Dump substr $x, 50;" SV = PVLV(0x319e4c) at 0x3bf54 REFCNT = 1 FLAGS = (PADMY,GMG,SMG,pPOK) IV = 0 NV = 0 PV = 0x2f42dc "\0\0\0...\0\0\0"\0 <----- CUR = 50 LEN = 52 MAGIC = 0x327f94 MG_VIRTUAL = &PL_vtbl_substr MG_TYPE = PERL_MAGIC_substr(x) TYPE = x TARGOFF = 50 TARGLEN = 50 TARG = 0x2f875c SV = PV(0x36034) at 0x2f875c REFCNT = 2 FLAGS = (POK,pPOK) PV = 0x330f6c "\0\0\0...\0\0\0"\0 CUR = 100 LEN = 104

$ref continues to refer (directly) to the memory allocate to $string.

Neither $ref nor $$ref ever refer directly to $string's PV. They can't because the address of $string's buffer can change as $string changes.

This would require retaining a pointer back to 'parent' string with the lvalue ref

$$ref does have a reference to $string.

Probably difficult to orchestrate.

Actually, that's easy. Just weaken $$ref's reference to $string.

At which point the extraneous (pre and/or post fix) bits of $string are GC'd leaving $ref pointing at just that which it references.

One can't free the start of a memory block. I don't think one can even free the end of a memory block. The string would have to be copied to a new buffer in order to shrink the buffer. Doable, but it would require a temporary "doubling" of memory.

It would also be uncharacteristic of Perl. Perl intentially avoids freeing memory left and right. Shrinking a buffer would be a first!

Replies are listed 'Best First'.
Re^8: Tracking down an Lvalue bug?
by BrowserUk (Patriarch) on Mar 23, 2012 at 01:14 UTC
    ActivePerl 5.14.0 (32 bit) on Windows

    Confirming that the behaviour is also changed on 64-bit 5.14 for windows.

    There will be a memory increase if you read the string $$r, but that's how magic works in Perl.

    I accept that is how it currently works, but question whether is should or has to work that way.

    If the reference is only ever used for reading, duplicating the reference memory is a complete waste of time (cpu & memory).

    At the very least it should be possible to defer the duplication until a mutating operation is performed upon it.

    Even then, I question whether it is necessary to duplicate the substring, and then copy the modified version back over the original.

    If a temporary SV* (be it a TARG or LVTARG or just a mortal SV* created for the purpose) was initialised so that its PV pointed directly at the original strings PV buffer, and it used the OOK trick, CUR & LEN to make that temporary SV* point to just the referenced substring in-place. And then that temp SV was passed to the mutating operator, everything should work as it would with a normal SV.

    When the mutating operator completes, the PUT magic gets control and can fix up the original (referenced) Sv to reflect any changes.

    But, I am aware that this is the kind of thing we could argue back forth, round & round and down a dozen blind alleys and never reach a conclusion. The only way to avoid that is to prove my theory. And that is going to take time.

    I'm also currently working on a machine that has a dodgy motherboard and an increasingly frequent habit of blue screening at the slightest provocation. I have a new motherboard and ram on its way, but till then, I'm not getting into anything deep because of the risk of loosing code though random failures.

    So bear with me. I'll either come back with proof of concept code or an acknowledgement that I couldn't make it work.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?