Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Why is the size even bigger after pack?

by PerlOnTheWay (Monk)
on Nov 03, 2011 at 02:51 UTC ( [id://935562]=perlquestion: print w/replies, xml ) Need Help??

PerlOnTheWay has asked for the wisdom of the Perl Monks concerning the following question:

use Devel::Size 'total_size'; my $num=1000; print total_size($num) . $/; print total_size(pack("w*",$num)) . $/;

the output is:

24 48

why?

update

the result become

48 48

if I put it into a file

Replies are listed 'Best First'.
Re: Why is the size even bigger after pack?
by grantm (Parson) on Nov 03, 2011 at 04:37 UTC
    Devel::Size it telling you how many bytes are taken up by the C structure(s) that are used to represent your Perl variable. The numbers you're getting suggest it takes 24 bytes to represent a scalar containing an integer and 48 bytes to represent a scalar containing a string. I can't say I find that particularly surprising.

      What's the point of pack if it takes more memory after packing?

        What did you think it would do?

        pack turns a list of values into a string according to a specific format. If you want to interact with something else which expects values in a specific format (a system call, a foreign function, a binary protocol), pack is your tool. If you want to use less memory, you need something else.


        Improve your skills with Modern Perl: the free book.

        Your example only shows one single case. An SV can hold IV's as well as PV's at the same time and a PV representation of 1 takes less space than that of big numbers. YMMV (even with different versions of perl):

        $ cat test.pl use v5.12; use Devel::Size "total_size"; for my $num (1, 1000, 100_000_000) { say "Value $num:"; say total_size ($num); say total_size (pack "w*", $num); } for my $num ("1", "1000", "100_000_000") { say "Value $num:"; say total_size ($num); say total_size (pack "w*", $num); } $ perl test.pl Value 1: 64 48 Value 1000: 64 48 Value 100000000: 72 48 Value 1: 48 48 Value 1000: 48 48 Value 100_000_000: 56 48 $

        Enjoy, Have FUN! H.Merijn
Re: Why is the size even bigger after pack?
by ikegami (Patriarch) on Nov 03, 2011 at 08:15 UTC
    use feature qw( say ); use Devel::Size qw( total_size ); use Devel::Peek qw( Dump ); my $num = 1000; say total_size($num); Dump($num); say ''; my $packed = pack("w*",$num); say total_size($packed); Dump($packed);

    5.12, 64 bit

    24 SV = IV(0x768010) at 0x768018 <-- See SVt_IV image below REFCNT = 1 FLAGS = (PADMY,IOK,pIOK) IV = 1000 <-- 8 bytes 48 SV = PV(0x752f28) at 0x768000 <-- See SVt_PV image below REFCNT = 1 Three more fields than SVt_IV FLAGS = (PADMY,POK,pPOK) PV = 0x7d6520 "\207h"\0 CUR = 2 <-- Even though only 3 are needed, LEN = 8 <-- 8 byte string buffer allocated

    SVt_IV, SVt_PV

    Are the greyed out fields outside of allocated memory or simply unused? [ See dave_the_m's reply ]

    (The numbers still don't quite add up, but it should give you an idea.)


    5.14, 32 bit

    16 SV = IV(0x51a100) at 0x51a104 <-- See SVt_IV image below REFCNT = 1 FLAGS = (PADMY,IOK,pIOK) IV = 1000 <-- 4 bytes 36 SV = PV(0x5e8cc4) at 0x51a054 <-- See SVt_PV image below REFCNT = 1 Four more fields than SVt_IV FLAGS = (PADMY,POK,pPOK) PV = 0x5b4334 "\207h"\0 CUR = 2 <-- Even though only 3 are needed, LEN = 12 <-- 12 byte string buffer allocated

    SVt_IV, SVt_PV

    (The numbers still don't quite add up, but it should give you an idea.)

    Update: Adjusted based on dave_the_m's explanation.

      In the case of the IV, the greyed-out bits represent memory that doesn't actually exist. The IV value is actually stored in the svu_pv slot in the head, and the sv_any pointer in the head actually points back to itself (with a slight offset), so that it appears to be pointing to a struct whose fourth field is an xiv_iv. In this way only a head and not a body needs allocating, but all the perl functions which expect to find it as the fourth field of the body still work.

      Dave.

        The IV value is actually stored in the svu_pv slot in the head

        That's what I thought!

        Thanks for the explanation, that clears up a lot. What doesn't illguts mention any of this!?

      I think it's simply unused.
        Based on what?

      I get different result when I run it with command line perl -we 'xxx' and perl filename

      what about you?

        No. Show the difference
Re: Why is the size even bigger after pack?
by flexvault (Monsignor) on Nov 03, 2011 at 13:01 UTC

    I read this yesterday, and couldn't understand the question.

    This is a guess, but maybe the Title should have been "How do I compress a string variable in perl?"

    To compress strings and save memory/disk, you can use the core modules IO::Compress::Gzip and IO::Compress::Gunzip The examples are good, so I don't think you need a code sample. In working with HTML, I have gotten 10x to 20x string reductions. For caching data, that's great!

    But what can you do with 'pack/unpack'? -- Lots of great things. These commands greatly reduce the complexity of transferring binary data from big-endian to little-endian machines and vice-versa. And how would you work with 'sockets'/'DNS' etc. without them.

    But I just found a new use (for me). I do a lot of work with public/private encryption keys. If the data gets corrupted, you need to generate new keys. Even worse, what if the keys have been changed on purpose.

    What I do now is the following ( non-verified code sample ):

    use String::CRC32; ## you can use 64 bit as well, but then need to ch +ange pack/unpack functions ## May need to install from CPAN, but has installe +d on all systems I use my $keys = pack( "N N",length( $PublicKey ), length( $PrivateKey ) ) . + $PublicKey . $PrivateKey; my $crc = crc32($keys); my $record = pack("N N", $crc , length($keys) ) . $keys;

    Keep a copy of this file on a non Internet connected computer and before the business day starts, the data is verified for accuracy of the keys.

    I for one, am glad perl has 'pack/unpack'!

    Thank you

    "Well done is better than well said." - Benjamin Franklin

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://935562]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (9)
As of 2024-03-28 09:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found