in reply to Re^3: memory use array vs ref to array
in thread memory use array vs ref to array

You don't need the do if you don't use the value of the last expression.
my $bigstring; { local $/; $bigstring = <FILE>; }

Update: Moreover, I tried both ways on a 2GB file and didn't notice any difference in memory consumption:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ C +OMMAND 7164 choroba 20 0 1970364 1.858g 3508 R 48.00 24.04 0:00.48 p +erl 7164 choroba 20 0 1970364 1.866g 3508 S 0.990 24.15 0:00.49 p +erl 7164 choroba 20 0 1970364 1.866g 3508 S 0.000 24.15 0:00.49 p +erl 7164 choroba 20 0 1970364 1.866g 3508 S 0.000 24.15 0:00.49 p +erl 7164 choroba 20 0 1970364 1.866g 3508 S 0.000 24.15 0:00.49 p +erl 7166 choroba 20 0 1970364 1.866g 3564 S 48.51 24.15 0:00.49 p +erl 7166 choroba 20 0 1970364 1.866g 3564 S 0.000 24.15 0:00.49 p +erl 7166 choroba 20 0 1970364 1.866g 3564 S 0.000 24.15 0:00.49 p +erl 7166 choroba 20 0 1970364 1.866g 3564 S 0.000 24.15 0:00.49 p +erl 7166 choroba 20 0 1970364 1.866g 3564 S 0.000 24.15 0:00.49 p +erl

Code run:

perl -wE 'my $string = do { local $/; <> }; say length $string; sleep +5' 2g perl -wE 'my $string ; { local $/; $string = <> }; say length $string; + sleep 5' 2g

($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

Replies are listed 'Best First'.
Re^5: memory use array vs ref to array
by BrowserUk (Patriarch) on Sep 17, 2016 at 21:41 UTC
    You don't need the do if you don't use the value of the last expression.

    True. But it's habitual.

    Moreover, I tried both ways on a 2GB file and didn't notice any difference in memory consumption:

    I'm not familiar with the ins and outs of memory measurement on *nix; but I can demonstrate the difference on Windows.

    C:\test>p1 [0]{} Perl> print mem;; 9,432 K []{} Perl> open I, '<', 'test.dat'; my $s; do{ local $/; $s = <I> }; p +rint mem;; 997,784 K []{} Perl> Terminating on signal SIGINT(2) C:\test>p1 [0]{} Perl> print mem;; 9,440 K []{} Perl> open I, '<', 'test.dat'; my $s = do{ local $/; <I> }; print + mem;; 1,986,060 K []{} Perl> Terminating on signal SIGINT(2)

    As you can see, in the latter case, the memory assigned to the process is double: ( 997784 - 9432 ) * 2 + 9440 = 1986146; almost exactly the 1,986,060 K measured in the latter case.

    The reason is that in the later case, the data is read into an internal mortal temporary scalar; and then copied from there to the named lexical, before the memory attached to the temp is freed. As the allocation is greater than (from memory) 1MB, (on windows at least) such huge allocations are allocated directly from the OS's virtual memory rather than from the process' heap; and then get released directly back to OS.

    Which is usually a good thing, but it still means you need to have double the memory available for a short while, and if you are close to the limits, that can blow the process.

    Is it possible that large allocations are also freed back to the OS on *nix, and you are measuring after it has been freed?


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
      > Is it possible that large allocations are also freed back to the OS on *nix, and you are measuring after it has been freed?

      I'm not sure where this mem comes from. I tried with Memory::Usage, measuring the consumption with top running with 0.001s delay (was probably more in practice):

      #!/usr/bin/perl use warnings; use strict; use feature qw{ say }; use Memory::Usage; my $file = shift; open my $FH, '<', $file or die $!; my $string; my $mu = 'Memory::Usage'->new; $mu->record('start'); { local $/; $string = <$FH> }; $mu->record('after do'); undef $string; seek $FH, 0, 0; $string = do { local $/; <$FH> }; $mu->record('after no do'); $mu->dump;

      Output:

      time vsz ( diff) rss ( diff) shared ( diff) code ( diff) + data ( diff) 0 17504 ( 17504) 3916 ( 3916) 3328 ( 3328) 1644 ( 1644) + 1060 ( 1060) start 0 1970632 ( 1953128) 1957112 ( 1953196) 3520 ( 192) 1644 +( 0) 1954188 ( 1953128) after do 1 1970632 ( 0) 1957248 ( 136) 3520 ( 0) 1644 ( + 0) 1954188 ( 0) after no do

      Top output:

      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

        It looks like you've got your "after do" and "after no do" labels the wrong wayaround; but I agree, it looks like there is no difference on your version of Perl on *nix.

        I was running under my default perl5.10; so I re-ran the tests under the latest version I have installed, 5.22 and I see the same results:

        C:\test>\Perl22\bin\p1.pl [0]{} Perl> print mem;; 9,404 K []{} Perl> open I, '<', 'test.dat'; my $s; do{ local $/; $s = <I> }; p +rint mem;; 997,760 K []{} Perl> Terminating on signal SIGINT(2) C:\test>\Perl22\bin\p1.pl [0]{} Perl> print mem;; 9,392 K []{} Perl> open I, '<', 'test.dat'; my $s = do{ local $/; <I> }; print + mem;; 1,986,016 K []{} Perl> Terminating on signal SIGINT(2)

        So maybe it's 'a windows thing'.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm not sure where this mem comes from.

        BTW. I missed/ignored that at first reading. mem() is a function defined in my REPL:

        sub mem{ `tasklist /nh /fi "PID eq $$"` =~ m[(\S+ K)$] }

        And tasklist is a system utility, which with the above parameters produces:

        C:\test>\Perl22\bin\p1 [0]{} Perl> print `tasklist /nh /fi "PID eq $$"`;; perl.exe 2524 Console 1 +9,384 K

        And without the /n{o}h{eader} switch gives:

        [0]{} Perl> print `tasklist /fi "PID eq $$"`;; Image Name PID Session Name Session# Me +m Usage ========================= ======== ================ =========== ===== +======= perl.exe 2524 Console 1 +9,428 K

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.
        Please try
        sub psMem { my( $pid ) = @_; my @lines = qx{$GOTPS -o vsize,rss -p $pid }; chomp @lines; foreach my $line (@lines) { my ( $vsize, $rss) = split /\s+/, $line; return "VM: $vsize RSS: $rss "; } return; }