Re^2: Memory efficient way to deal with really large arrays?

That's brilliant. It's simple and very efficient.

Another approach would have been to pack the pairs into a 64 bit string to be stored in one array, that might be more efficient but wouldn't be as simple as this trick.

Cheers Rolf
_{(addicted to the Perl Programming Language :)

Wikisyntax for the Monastery}

Comment on Re^2: Memory efficient way to deal with really large arrays?

Replies are listed 'Best First'.
Re^3: Memory efficient way to deal with really large arrays? by salva (Canon) on Dec 13, 2020 at 13:21 UTC
that might be more efficient I am not sure. Some years ago Nicholas Clark introduced a change in the internal representation of SVs that made then very efficient for integers and doubles. Briefly, all internal types used to have a fixed size head (SV_HEAD) and a variable sized body. That change, used a trick to embed the body inside the header for SVs representing numbers, making then more cache friendly and reducing memory usage by eliminating the body part. So, having two arrays with numbers may actually use less memory than an array of small strings.	[reply]

Replies are listed 'Best First'.

Re^3: Memory efficient way to deal with really large arrays?
by salva (Canon) on Dec 13, 2020 at 13:21 UTC

that might be more efficient

I am not sure. Some years ago Nicholas Clark introduced a change in the internal representation of SVs that made then very efficient for integers and doubles. Briefly, all internal types used to have a fixed size head (SV_HEAD) and a variable sized body. That change, used a trick to embed the body inside the header for SVs representing numbers, making then more cache friendly and reducing memory usage by eliminating the body part.

So, having two arrays with numbers may actually use less memory than an array of small strings.

[reply]