szabgab has asked for the wisdom of the Perl Monks concerning the following question:

Given the following script: 
$mem = "A" x 1024 ** 3;

[download]
using htop I see the script is uing 2Gb memory.

Am I missing something or is that Perl created the 1Gb string in memory and then copies it tho $mem and does not release the original? With the emphasize on "copy".

Further "research" shows that this code reports the usage of 3Gb memory and swapped out my browser so it is now very slow to type :). 

$mem = "A" x 1024 ** 3;
$t = $mem;

[download]

Replies are listed 'Best First'.
Re: memory usage
by CountZero (Bishop) on Jan 16, 2012 at 13:17 UTC
    Perl is indeed reluctant to release "unused" memory. Of course, your definition of "unused" may be different from Perl's.

    In your second script, you indeed need at least 2GB to keep the data in $t and $mem.

    And you are right that in your first script, Perl first builds the string and then copies it to the variable. After this assignment, the memory used to build the string can be re-used, but Perl rather reserves another batch of memory when you copy the first variable. I *think* it is because the memory footprint of the variable is slightly more than the memory used for the building of the string.

    Consider the following two scripts: 

    $mem1 = 'a' x 1024 ** 2;
$mem2 = 'b' x 1024 ** 2;

    [download]
    and 
    $mem1 = 'a' x 1024 ** 2;
$mem2 = 'b' x 1024 ** 2;
$mem1 = $mem2;

    [download]
    Both scripts use almost exactly the same amount of memory, but of course, both variables will perfectly fit into each other.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

[reply]
[d/l]
[select]
Re: memory usage
by BrowserUk (Patriarch) on Jan 16, 2012 at 17:17 UTC

    If you need to allocate huge scalars, do it this way instead:

    my $x = 'A'; $x x= 1024**3;

    [download]

    And you'll use just the gigabyte plus whatever you already had.

    Also, it happens about 3 times faster even if your original didn't move you into swapping. And many times faster if it did.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?
[reply]
[d/l]
Re: memory usage
by Anonymous Monk on Jan 16, 2012 at 12:43 UTC
    FWIW, creating an explicit scope and dealing with only a reference to the data does not help, still uses 2GB, with and without usemymalloc. 
    use GTop qw();
my $gtop = GTop->new;
my $before = $gtop->proc_mem($$)->size;

my $mem;
{
    $mem = \("A" x 1024 ** 3);
}

my $after = $gtop->proc_mem($$)->size;

print GTop::size_string($after - $before);

    [download]
[reply]
[d/l]
Re: memory usage
by locked_user sundialsvc4 (Abbot) on Jan 16, 2012 at 17:07 UTC

    Dealing with a monolithic object of gigabyte size is going to be a strain on any memory management system (including the OS’s ...) so if at all possible you want to avoid doing that.   You have just allocated and written to nearly a quarter-million 4K pages ... no wonder the system is not happy to see you.

Back to Seekers of Perl Wisdom