I use TinyPerl 5.8 on Windows, and I noticed that when I create a large variable, the size of the process expands, but when I use undef $variable, then the size of the process shrinks (but not necessarily by the same amount! lol).
I noticed that if I do not pass by reference, then an extra copy of the string is created in memory. I see that because I have the Task Manager open on the side, and the memory usage of the TinyPerl.exe application jumps to twice the expected size. But of course, I think, that's normal, and that's how it should work.
Some unexpected behavior happens when I create a variable and initialize it like so: my $VAR1 = 'A' x 10000000;
In this case, TWICE the amount of memory is used! However, if I initialize it in two steps using the vec() function, then only the exact amount of memory is used:
my $VAR2 = ''; vec($VAR2, 9999999, 8) = 65;
The latter however fills the string with zero bytes and puts a letter 'A' at the end, while the former fills it with letter A's all the way. I could use vec() in a for loop to fill the variable with letter A's, but that takes MUCH longer than using the 'x' operator.
Also, when it's time to get rid of the variables and I do undef $VAR1; and undef $VAR2; an unexpected thing happens! undef $VAR1; releases only half of the memory, while undef $VAR2; releases the entire amount. So, when using the 'x' operator to create a large variable, not only does that take up twice as much memory, but when I undef it, it does not release all of the used memory, which looks like a memory leak in TinyPerl 5.8. I don't know how Perl in Linux and MacOS handle these two scenarios, but it's very suspicious the way it is handled in Windows.
I have tested how storing large strings in individual variables differs from storing them in a large array, and it seems that that makes no difference in memory usage. I did notice however that storing MANY short strings (let's say a million 8-byte strings) is handled and stored a lot more efficiently when it's all packed into ONE string as opposed to an array with one million elements each containing an 8-byte string. So, when someone deals with lots of tiny pieces, it's better to store them into a string than have a giant array.
I used this test program:
#!/usr/bin/perl use strict; use warnings; $| = 1; print "\nSTEP 1: Reserve memory\n"; #my @GLOBAL_DATA; #$GLOBAL_DATA[0] = ''; #vec($GLOBAL_DATA[0], 9999999, 8) = 44; # Reserves just the right amou +nt. #$GLOBAL_DATA[1] = 'x' x 10000000; # Reserves 20000000 bytes #$GLOBAL_DATA[2] = 'y' x 100000; # Reserves 200000 bytes #$GLOBAL_DATA[3] = 'z' x 1000; # Reserves 2000 bytes my $AA = 'H'; vec($AA, 9999999, 8) = 44; # Reserves just the right amount. #my $BB = 'g' x 10000000; # Reserves double memory!!! $a = <STDIN>; print "\nSTEP 2: Do something with the data\n"; DoSomething(\$AA); # Pass by reference. PrintIt(\$AA); # Pass by reference. $a = <STDIN>; print "\nSTEP 3: Free up memory\n"; undef $AA; # Frees up all of it #undef $BB; # Frees up only half of it! #undef $GLOBAL_DATA[0]; # Frees up all of it #undef $GLOBAL_DATA[1]; # Frees up only half of it! #splice(@GLOBAL_DATA, 1, 1); # Frees up half $a = <STDIN>; print "\nSTEP 4: Pause\n$a"; exit; sub DoSomething { my $X = $_[0]; for (my $i = 0; $i < 60; $i++) { vec($$X, $i, 8) = $i + 48; # This operation doesn't use more m +emory } for (my $i = 9990000; $i < 9999999; $i++) { vec($$X, $i, 8) = 65; # This operation doesn't use more memory } print "\nDONE."; } sub PrintIt { my $X = $_[0]; print "\nLENGTH OF STRING = ", length($$X), "\n"; print "\nPREVIEW = ", substr($$X, 0, 60), "\n"; }
After I initialize the $BB variable using the 'x' operator, I opened HxD memory viewer in Windows to look into TinyPerl's data segment, and as I scrolled through all the stuff, I found a large region of memory from 024f0020 to 038096B0 which is filled with letter g's. That's approximately 20,000,000 bytes. So, why does Perl create double the amount of letters when using the 'x' operator???? This is so weird.
In JavaScript, for example, every string is stored in Unicode format, so one character takes up 2 bytes in memory. So, if I create a 10 MB string, that will normally take up double the space in memory. And if I look into it using HxD hex viewer, I'd see that its data segment is filled with 00 67 00 67 00 67 00 67 00 67 which is normal. That's how string of "gggggggggg" is stored by JavaScript. That's what I thought Perl does too (storing characters in unicode format), but that's not the case! It literally creates a string that is twice the size.
In reply to Re^2: Memory efficient design
by harangzsolt33
in thread Memory efficient design
by harangzsolt33
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |