Re^9: memory use array vs ref to array

:) whoops

use File::Which qw/ which /;
our $GOTPS = which('ps');
[download]

I've used the PID reported by the script in another terminal and ran:

groan :) thats too much lines

#!/usr/bin/perl --
use warnings;
use strict;
use feature qw{ say };
use File::Which qw/ which /;
our $GOTPS = which('ps');

say $$;
psMem();

my $file = shift;
open my $FH, '<', $file or die $!;

my $string;

{ local $/; $string = <$FH> };
say length $string;

psMem(); 

undef $string;
psMem(); 

seek $FH, 0, 0;
$string = do { local $/; <$FH> };
say length $string;

psMem();
undef $string;
psMem(); 


sub psMem {
    my( $pid ) = @_;
    my @lines = qx{$GOTPS   -o vsize,rss     -p $pid };
    chomp @lines;
    foreach my $line (@lines) {
        my ( $vsize, $rss) = split /\s+/, $line;
        return "VM: $vsize   RSS: $rss  ";
    }
    return;
}
[download]

Comment on Re^9: memory use array vs ref to array Select or Download Code

Replies are listed 'Best First'.
Re^10: memory use array vs ref to array by choroba (Cardinal) on Sep 19, 2016 at 07:55 UTC
You don't send any parameter to `psMem` . Also, you never output anything related to memory consumption. But, most importantly, it can still miss the allocation, as BrowserUk say the memory "gets released directly back to OS": > The reason is that in the later case, the data is read into an internal mortal temporary scalar; and then copied from there to the named lexical, before the memory attached to the temp is freed. As the allocation is greater than (from memory) 1MB, (on windows at least) such huge allocations are allocated directly from the OS's virtual memory rather than from the process' heap; and then get released directly back to OS. After having fixed the code, I ran your code with the following output: `6378 VM: RSS: VSZ VM: RSS: 17780 2000000000 VM: RSS: VSZ VM: 1970908 RSS: 1957712 VM: RSS: VSZ VM: RSS: 17780 2000000000 VM: RSS: VSZ VM: 4412316 RSS: 3910984 VM: RSS: VSZ VM: 1970908 RSS: 1957856` [download] Wait, it seems the memory consumption is indeed doubled! I checked with my previous code to realize I was running it on a different machine with different Perl version. I tried with blead, too, and it seems memory is doubled on Linux in 5.18, but not in 5.20 and later. ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^10: memory use array vs ref to array
by choroba (Cardinal) on Sep 19, 2016 at 07:55 UTC

psMem

BrowserUk

> The reason is that in the later case, the data is read into an internal mortal temporary scalar; and then copied from there to the named lexical, before the memory attached to the temp is freed. As the allocation is greater than (from memory) 1MB, (on windows at least) such huge allocations are allocated directly from the OS's virtual memory rather than from the process' heap; and then get released directly back to OS.

After having fixed the code, I ran your code with the following output:

6378
VM:    RSS: VSZ  
VM:    RSS: 17780  
2000000000
VM:    RSS: VSZ  
VM: 1970908   RSS: 1957712  
VM:    RSS: VSZ  
VM:    RSS: 17780  
2000000000
VM:    RSS: VSZ  
VM: 4412316   RSS: 3910984  
VM:    RSS: VSZ  
VM: 1970908   RSS: 1957856
[download]

Wait, it seems the memory consumption is indeed doubled! I checked with my previous code to realize I was running it on a different machine with different Perl version. I tried with blead, too, and it seems memory is doubled on Linux in 5.18, but not in 5.20 and later.

($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
[download]

[reply]
[d/l]
[select]