theantler has asked for the wisdom of the Perl Monks concerning the following question:

Dear folks .. I had a cracy (?) thought about pack/unpack .. Could we pack a hash? Or a complex data structure, like a record with multiple fields of different types: Fx. a long, a timestamp (also a long?), an array, and a hash of arrays. Would that difficult or impossible to do, and is there a reason not try pack such a thing? Some general sense would be very appreciated. Ta

Replies are listed 'Best First'.
Re: Packing hashes or complex structures?
by Anonymous Monk on Mar 23, 2010 at 10:46 UTC
    Could we pack a hash?

    Sure

    Would that difficult or impossible to do,

    not difficult or impossible

    and is there a reason not try pack such a thing?

    Might be an interesting toy, but these are wheels for that already (ex: Storable, MLDBM, DBD::SQLite)

      MLDBM is obsolete, use DBM::Deep instead. It allows direct access to arbitrarily-deeply nested structures, without having to pull huge chunks of the structure into memory first.
        MLDBM is obsolete, use DBM::Deep instead. It allows direct access to arbitrarily-deeply nested structures, without having to pull huge chunks of the structure into memory first.

        No thanks, i'd reather use Storable or SQLite

Re: Packing hashes or complex structures?
by jethro (Monsignor) on Mar 23, 2010 at 12:51 UTC

    First we would have to answer WHY we would want to pack a hash (You probably meant 'pack' as in 'compress' or 'zip', right? The perl function pack has a different purpose than compression).

    It would have the following advantage:

    1. You could keep those few applications unchanged that are so big that they don't fit into memory anymore but that would still fit if compressed by about 5 to 50% (the usual compression rate of general compression algorithms, assuming you even could reach that with a hash compression, which is very doubtful). Most people would tell you to buy more RAM instead or adapt your algorithm/data structure

    ... and the following disadvantages

    1. A compressed hash isn't a hash anymore, before accessing it you would have to uncompress it. You couldn't use $hash{'bla'} anymore. So you add a lot of inconvenience to the handling of hashes for the programmer

    2. Uncompressing takes time. More than the time it takes to access a hash, so we are talking about a speed penalty of more than 100%. For a few free bytes more the hashes are seriously slower now.

    3. Compressing algorithms don't guarantee compression. So even if you had a program that just needed 5% compression to fit into memory you never could be sure that your compressed hash could deliver it. With the "wrong" data your compression rate could be 0.

    4.

Re: Packing hashes or complex structures?
by JavaFan (Canon) on Mar 23, 2010 at 15:51 UTC
    Could we pack a hash?
    Well, yes. pack takes a format and a list of values, and it's rather trivial to turn a hash into a list of values (although you probably want a deterministic order). You probably want something like (untested):
    my %hash = make_me_a_hash; my @key_order = qw[key1 key2 key3 key4]; my $packformat = "ABCDEFGH"; my $packed_str = pack $packformat, map {$_, $hash{$_}} @key_order;
    And you should be able to unpack that as:
    my %hash = unpack $packformat, $packed_str;
      That template does not look valid .. But, thats a cool way to do it with map and spitting out a list, I had not though it that way. I am going to try that! Thanks folks ..