Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: A memory efficient hash, trading off speed - does it already exist?

by dragonchild (Archbishop)
on Feb 03, 2003 at 23:34 UTC ( [id://232412]=note: print w/replies, xml ) Need Help??


in reply to A memory efficient hash, trading off speed - does it already exist?

Abigail-II mentioned that you should reduce the number of hashes you are using. Another option is to use arrays instead of hashes. You can do some funkiness under the hood to try and maximize each array (as an array pre-allocates to like 50 elements at a time).

One option is to create an object that uses an array as its implementation. I have a version and I'm sure that CPAN does, as well. You can go ahead and make it one big array for all the objects, having the class keep track of where each object starts in the super-array. There's a bit of funkiness to it, but it's definitely do-able. (/msg me if you want more info)

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Replies are listed 'Best First'.
Re: Re: A memory efficient hash, trading off speed - does it already exist?
by JPaul (Hermit) on Feb 04, 2003 at 04:21 UTC
    I tried this earlier on when it became apparent the project would be sucking up great quantities of memory - with very little success. In fact, I think it changed the memory usage by less than a few hundred kilobytes...
    This was some time ago, the data set has changed greatly. Perhaps I should try this again and would benefit more from it with a more diverse data set.

    I don't even begin to understand that second paragraph. It amazes me how much I can do in perl, and how remarkably little I truly understand about the workings of it all. How embaressing.

    My thanks,
    JP
    -- Alexander Widdlemouse undid his bellybutton and his bum dropped off --

      Let's say you know you will be making a hash of 120,000 hashes, each with (up to) 2 values. Instead of making 120,000 tiny hashes, with the overhead of each hash, you can go ahead and make 120,000 blessed scalar-refs and one huge array of 360,000 scalars. Each object will be a blessed scalar. Something like (untested!):
      package CoolObject; my @objects; my $next_index = 0; sub new { my $class = shift; my $x = $next_index; my $self = bless \$x, $class; $next_index += 3; return $self; } sub name { my $self = shift; $object[$$self] = shift if @_; return $object[$$self]; } sub attr1 { my $self = shift; $object[$$self + 1] = shift if @_; return $object[$$self + 1]; } sub attr2 { my $self = shift; $object[$$self + 2] = shift if @_; return $object[$$self + 2]; } 1;
      Those having read the Panther book will recognize this. Some obvious improvements would include:
      1. Adding the kind of garbage collection he had to re-use entries. (This would be in DESTROY.)
      2. Initialization of attributes. (This would be in new)
      3. Making it into more of a base class that will take attributes. (This would involve adding some sort of attribute registration.)
      4. Adding attribute inheritance. (This would involve attribute registration that walks the ISA-tree.)

      ------
      We are the carpenters and bricklayers of the Information Age.

      Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://232412]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (7)
As of 2024-03-28 10:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found