Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re^2: Memory efficient way to deal with really large arrays?

by LanX (Sage)
on Dec 13, 2020 at 13:01 UTC ( #11125104=note: print w/replies, xml ) Need Help??


in reply to Re: Memory efficient way to deal with really large arrays?
in thread Memory efficient way to deal with really large arrays?

That's brilliant. It's simple and very efficient.

Another approach would have been to pack the pairs into a 64 bit string to be stored in one array, that might be more efficient but wouldn't be as simple as this trick.

Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery

  • Comment on Re^2: Memory efficient way to deal with really large arrays?

Replies are listed 'Best First'.
Re^3: Memory efficient way to deal with really large arrays?
by salva (Canon) on Dec 13, 2020 at 13:21 UTC
    that might be more efficient

    I am not sure. Some years ago Nicholas Clark introduced a change in the internal representation of SVs that made then very efficient for integers and doubles. Briefly, all internal types used to have a fixed size head (SV_HEAD) and a variable sized body. That change, used a trick to embed the body inside the header for SVs representing numbers, making then more cache friendly and reducing memory usage by eliminating the body part.

    So, having two arrays with numbers may actually use less memory than an array of small strings.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11125104]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (4)
As of 2022-05-21 13:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (76 votes). Check out past polls.

    Notices?