Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Memory efficient way to deal with really large arrays?

by LanX (Saint)
on Dec 13, 2020 at 00:17 UTC ( [id://11125082]=note: print w/replies, xml ) Need Help??


in reply to Memory efficient way to deal with really large arrays?

> push @p, [ $p1, $p2];

I guess that creating 500e06 sub-arrays is the most memory consuming part.

I can't tell how efficient the usage of @d is, because it depends on how sparse the entries are used.

32 bits means 4.2e9 potential entries but with 500e6/256 = 1953125 in worst case you'll end up with 2200 empty entry ratio in @d. I bet a hash is more memory efficient then.

But it really depends what you are trying to achieve with all that data.

update

> which don't occur more than 256 times each.

your "test" doesn't reflect that.

Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11125082]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (7)
As of 2024-04-23 18:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found