Question on SV internals

John M. Dlugosz has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Question on SV internals by davido (Cardinal) on May 08, 2011 at 04:53 UTC
Use the Devel::Peek module's Dump() function to inspect the various compartments within a scalar at whatever stage you wish in the script. Example: `use Devel::Peek; my $x = "99 bottles of beer on the wall"; Dump( $x ); $x += 1; Dump( $x ); $x .= "\n"; Dump( $x ); $x += 2; Dump( $x );` [download] Have fun exploring what various actions do to your scalar internally. I can't remember where I read a good explanation of the internals of Perl's various data types. It might have been OReilly's Advanced Perl Programming... or it might not. Hopefully someone can help out with a good reference. PS: To specifically answer the question; once Perl has recognized it in numeric context, it doesn't need to be re-'converted'. Dave	[reply] [d/l]
Re^2: Question on SV internals by Marshall (Canon) on May 08, 2011 at 05:12 UTC
Yes, you are correct, "Advanced Perl Programming", section 20 is "Perl Internals" and the diagrams are nice. I think there are other diagrams our there. OP might want to search for Perl guts or perlguts.	[reply]
Re^3: Question on SV internals by ikegami (Patriarch) on May 08, 2011 at 06:01 UTC
illguts	[reply]
Re: Question on SV internals by ikegami (Patriarch) on May 08, 2011 at 06:06 UTC
Yes. It becomes a dualvar-lite. Devel::Peek is very good at showing this. `>perl -MDevel::Peek -e"$_ = '20 ducks'; Dump($_); $_+0; Dump($_);" SV = PV(0xd888c) at 0x2be2ff4 REFCNT = 1 FLAGS = (POK,pPOK) PV = 0xd9acc "20 ducks"\0 CUR = 8 LEN = 12 SV = PVNV(0xda40c) at 0x2be2ff4 REFCNT = 1 FLAGS = (POK,pIOK,pNOK,pPOK) IV = 20 NV = 20 PV = 0xd9acc "20 ducks"\0 CUR = 8 LEN = 12` [download] POK = has a string IOK = has an signed int NOK = has a floating point number If there a "p" flag an no corresponding non-"p" flag, it's a generated value from numerical conversion or a temporary from magic. See also: illguts.	[reply] [d/l]
Re: Question on SV internals by davido (Cardinal) on May 08, 2011 at 05:54 UTC
Here's a really good blog-page on internal representation of Perl's scalars. It does confirm that once a string has had its numeric value computed it's saved so that it needn't be computed again. nothingmuch's perl blog: SV Dave	[reply]
Re: Question on SV internals by BrowserUk (Patriarch) on May 08, 2011 at 08:39 UTC
Now if I perform another numeric operation on that value, does it remember that it was 20 and not convert it again? Short answer: Yes, until you next use it in a string context. Ie. `@a = 1 .. 1e6;; print total_size \@a;; 32000176 "$_" for @a;; Useless use of string in void context at (eval 11) print total_size \@a;; 72000176 @a = 1 .. 1e6;; print total_size \@a;; 32000176 ''.$_ for @a;; Useless use of concatenation (.) or string in void context at (eval 15 +) print total_size \@a;; 72000176` [download] What the above shows is that apparently innocent and even useless references to numeric scalars in string contexts can cause a quite dramatic explosion in memory usage. Most of the time, individual scalars and small aggregates, this is inconsequential, but if you are dealing with large aggregates close to the limits of your memory, it can be worth your while to take care to prevent such conversions: `@a = 1 .. 1e6;; print total_size \@a;; 32000176 do{ my $n = $_; "$n" } for @a;; Useless use of string in void context at (eval 23) line 1, <STDIN> lin +e 15. print total_size \@a;; 32000176` [download] Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l] [select]
Re^2: Question on SV internals by ikegami (Patriarch) on May 08, 2011 at 15:54 UTC
Yes, until you next use it in a string context. There's no such terminating condition. `>perl -MDevel::Peek -e"$_ = '20 ducks'; Dump($_); 0+$_; Dump($_); ''.$ +_; Dump($_);" ... SV = PVNV(0x163a454) at 0x342be4 REFCNT = 1 FLAGS = (POK,pIOK,pNOK,pPOK) IV = 20 NV = 20 PV = 0x1639b14 "20 ducks"\0 CUR = 8 LEN = 12` [download] Another way to check is to count the number of numeric warnings: `>perl -wE"$_='20 ducks'; say 0+$_; say 0+$_; say ''.$_; say 0+$_;" Argument "20 ducks" isn't numeric in addition (+) at -e line 1. 20 20 20 ducks 20` [download]	[reply] [d/l] [select]
Re^3: Question on SV internals by BrowserUk (Patriarch) on May 08, 2011 at 16:42 UTC
`$s = '20 ducks';; Dump $s;; SV = PV(0x18aec0) at 0xa5b48 REFCNT = 1 FLAGS = (POK,pPOK) PV = 0x40cad18 "20 ducks"\0 CUR = 8 LEN = 16` [download] Length = 16. Now use it in a numeric constant. `$s += 1;; Argument "20 ducks" isn't numeric in addition (+) at (eval 13) line 1, + <STDIN> line 4. Dump $s;; SV = PVNV(0x40b1eb8) at 0xa5b48 REFCNT = 1 FLAGS = (NOK,pNOK) IV = 20 NV = 21 PV = 0x40cad18 "20 ducks"\0 CUR = 8 LEN = 16` [download] Hm. Length still 16, but we gained an IV = 20 and an NV = 21; so we gained 12 or 16 bytes. Oh, and the PV no longer reflects the value of the scalar, so when you use it in a string context, a conversion has to happen! `print $s;; 21 Dump $s;; SV = PVNV(0x40b1eb8) at 0xa5b48 REFCNT = 1 FLAGS = (NOK,POK,pNOK,pPOK) IV = 20 NV = 21 PV = 0x40d3068 "21"\0 CUR = 2 LEN = 40` [download] Which modifies the PV and oops, it grew to 40 bytes in the process. And I don't care how you try to weasel out of it, the evidence is right there for those that care to look. So don't bother. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l] [select]
Re^4: Question on SV internals by ikegami (Patriarch) on May 08, 2011 at 16:47 UTC
Re^5: Question on SV internals by BrowserUk (Patriarch) on May 08, 2011 at 19:09 UTC
Some notes below your chosen depth have not been shown here
Re: Question on SV internals by Tux (Canon) on May 08, 2011 at 18:35 UTC
With XS code you can even got to TripleVar SV's :) `$ perl -wl \ ? -e 'use charnames ":full";' \ ? -e 'use Data::Peek qw(triplevar DDump);' \ ? -e 'my $tv = triplevar ("\N{GREEK SMALL LETTER PI}", 3, 3.1415);' \ ? -e 'DDump $tv' SV = PVNV(0x1217860) at 0x76a528 REFCNT = 1 FLAGS = (PADMY,IOK,NOK,POK,pIOK,pNOK,pPOK,UTF8) IV = 3 NV = 3.1415 PV = 0x7773e0 "\317\200"\0 [UTF8 "\x{3c0}"] CUR = 2 LEN = 8` [download] or in one line `perl -MData::Peek=triplevar,DDump -wle'DDump (triplevar ("\x{3c0}", 3, + 3.1415))' SV = PVNV(0x11d8f20) at 0x74b0e0 REFCNT = 1 FLAGS = (IOK,NOK,POK,pIOK,pNOK,pPOK,UTF8) IV = 3 NV = 3.1415 PV = 0x75b470 "\317\200"\0 [UTF8 "\x{3c0}"] CUR = 2 LEN = 8` [download] You could go even further by adding magic and controling the magic by a combination of the IV, NV, and PV parts. That however is sick :) Enjoy, Have FUN! H.Merijn	[reply] [d/l] [select]
Re: Question on SV internals by Marshall (Canon) on May 08, 2011 at 04:43 UTC
Your question is a bit confusing as you cannot do a numeric operation on a string like `"20 ducks in a row"` when warnings and use strict are in force. If the "20" was in a separate variable, once it is used in a numeric context, the string part of that variable doesn't matter anymore. What happens with leading zeroes illustrates this point... Below, $a starts out as a string with leading zeroes. Adding "0" to it (use in a numeric context), causes the "leading zeroes" to be eliminated, i.e. it is now a binary numeric value. The original string value of $a is now not accessible by any normal Perl operation (you won't know how many leading zeroes it had to begin with). If further math operations are done on $a, no further string to binary conversions are required. Perl math operations are very fast. Leading zeroes in a numeric assignment like shown below for $b, causes the numeric value to be interpreted as an octal number. `#!/usr/bin/perl -w use strict; my $a = '00000022'; my $b = 00000033; #leading zero means octal! print "$a $b\n"; #00000022 27 $a+=0; print "$a $b\n"; #22 27 #22 now numeric` [download] Update: -- extraneous verbage that added nothing deleted.	[reply] [d/l] [select]
Re^2: Question on SV internals by syphilis (Archbishop) on May 08, 2011 at 08:22 UTC
Your question is a bit confusing as you cannot do a numeric operation on a string like "20 ducks in a row" when warnings and use strict are in force No - you can do the numeric operation on that string if warnings and strict are enabled, and it gives the same result as you get without strict and warnings ... except, of course, that you additionally get a warning: `C:\>perl -Mstrict -Mwarnings -e "my $x = '20 ducks in a row'; $x += 17 +;print $x" Argument "20 ducks in a row" isn't numeric in addition (+) at -e line +1. 37 C:\>` [download] Cheers, Rob	[reply] [d/l]
Re^2: Question on SV internals by John M. Dlugosz (Monsignor) on May 09, 2011 at 21:52 UTC
I was thinking of taking a field that I'm originally using for a numeric value and adding some other (optional) flags to it. Making it a string comparison would give funny results for 4 vs 20, etc. Splitting it up into different fields first, or setting it up with additional (mostly unused) fields originally would be more work. I was assuming that using "20 ducks" as a numeric value just worked (albeit with a warning I can turn off in a tight scope). I would not expect the "ducks" to disappear, as I'll look at that part of the string later.	[reply]
Re^3: Question on SV internals by Tux (Canon) on May 09, 2011 at 22:05 UTC
The read the tutorial on `pack` and `unpack`. The advantage of `pack "NA20", 20, "ducks"` is that it sorts fantastic and fast and that the numeric part and the string part are well-defined separated from eachother. Enjoy, Have FUN! H.Merijn	[reply] [d/l] [select]