Memory Usage of Perl System command

pperi2 has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Memory Usage of Perl System command by Mr. Muskrat (Canon) on Jan 08, 2016 at 22:28 UTC
Could you please let me know how can I make sure that the memory doesn't increase Don't run `adb devices`? It is the source of the memory spike. Perl will hang on to allocated memory until the script terminates. The difference between the two numbers is how much memory that `adb devices` is using.	[reply]
Re^2: Memory Usage of Perl System command by Anonymous Monk on Jan 08, 2016 at 23:39 UTC
the thing is even if you run any command other than adb devices using system command, I am seeing the same memory increase. for example: replacing `system("adb devices");` [download] with `system("dir"); ## just an example, you can replace it with any comm +and` [download] the output looks like this: `########## memory usage of perl start of execution ########### perl.exe 17304 RDP-Tcp#0 1 9,536 K ############################################################## ########## memory usage of perl end of execution ########### perl.exe 17304 RDP-Tcp#0 1 68,372 K ##############################################################` [download] My question was how can we release the memory allocated by the system command? I understand that perl doesn't release the memory back to OS but it should re-use the memory rather than allocating new memory block for each call, isn't that right?	[reply] [d/l] [select]
Re^3: Memory Usage of Perl System command by Laurent_R (Canon) on Jan 09, 2016 at 00:27 UTC
And what do you get if you run the exact same thing, but just iterating once, 10 times, 100 times and perhaps 1,000 times, instead of 5,000 times on your system call? My gut-feeling idea is that the number of times you run your command is probably irrelevant to a large extent. Which would mean that you have a memory penalty for the first system call, but not really for the others, so that the memory usage would appear to be mostly reclaimed by the Perl process between the various calls. But that's only a guess, I don't really know and can't run the exact same tests. But relatively similar tests lead me to think that's probably what you will experience.	[reply]
Re^4: Memory Usage of Perl System command by pperi2 (Novice) on Jan 09, 2016 at 00:53 UTC
Re^3: Memory Usage of Perl System command by Mr. Muskrat (Canon) on Jan 08, 2016 at 23:47 UTC
I guess I should have tried to run it before commenting. I assumed it was `adb devices` doing it. BTW, I can't reproduce the problem on Linux.	[reply]
Re^4: Memory Usage of Perl System command by pperi2 (Novice) on Jan 09, 2016 at 00:24 UTC
Re: Memory Usage of Perl System command by Anonymous Monk on Jan 09, 2016 at 01:19 UTC
I'm running 5.16 I don't see leaks http://live.sysinternals.com/procexp.exe #!/usr/bin/perl -- sub mm { print pslistMem($$),"\n"; } sub pslistMem { my( $pid ) = @_; my @lines = qx{pslist -m $pid 2>NUL}; chomp @lines; #~ Name Pid VM WS Priv Priv Pk Faults +NonP Page #~ perl 1052 25348 7220 4504 4512 1977 + 2 33 my %ps; @ps{'Name', 'Pid', 'VM', 'WS', 'Priv', 'Priv Pk', 'Faults', 'NonP', 'Page' } = grep length, split /\s+/, $lines[-1]; #~ Pri Priority #~ Thd Number of Threads #~ Hnd Number of Handles #~ VM Virtual Memory #~ WS Working Set ------ "Mem Usage" in taskmanager #~ Priv Private Virtual Memory ------ "VM Size" in taskman +ager #~ Priv Pk Private Virtual Memory Peak #~ Faults Page Faults #~ NonP Non-Paged Pool #~ Page Paged Pool #~ Cswtch Context Switches #~ use DDS; Dump(\@lines , \%ps); return "WVM: $ps{VM} { WS: $ps{WS} VM: $ps{Priv} } "; } for ( 1 .. 5 ) { mm; for ( 1 .. 1000 ) { system 'echo >NUL'; } mm; } mm; __END__ WVM: 21648 { WS: 4476 VM: 1832 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } WVM: 21648 { WS: 4496 VM: 1844 } [download]	[reply] [d/l]
Re: Memory Usage of Perl System command by pperi2 (Novice) on Jan 09, 2016 at 22:16 UTC
Looks like the issue is seen only using perl 5.18.2, when I moved to different version of perl, the issue is not seen. Thanks for the responses.	[reply]
Re^2: Memory Usage of Perl System command by Anonymous Monk on Jan 10, 2016 at 22:23 UTC
Looks like the issue is seen only using perl 5.18.2, when I moved to different version of perl, the issue is not seen. Thanks for the responses. a `perl -V` from the bad version would be useful http://strawberryperl.com/download/5.18.2.2/strawberry-perl-5.18.2.2-32bit-portable.zip has no such problem :) `WVM: 19684 { WS: 3056 VM: 724 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 } WVM: 19684 { WS: 3128 VM: 736 }` [download]	[reply] [d/l] [select]
Re: Memory Usage of Perl System command by pperi2 (Novice) on Jan 09, 2016 at 01:10 UTC
as per your suggestion, I changed the code to get the memory usage for each iteration as shown below: ############################################################## ########## memory usage of perl start of execution for iter: 0 ####### perl.exe 18124 RDP-Tcp#0 1 9,544 K ############################################################## ########## memory usage of perl end of execution for iter: 0 ######## perl.exe 18124 RDP-Tcp#0 1 9,580 K ############################################################## iteration Completed : 0 ########## memory usage of perl start of execution for iter: 1 ####### perl.exe 18124 RDP-Tcp#0 1 9,588 K ############################################################## ########## memory usage of perl end of execution for iter: 1 ######## perl.exe 18124 RDP-Tcp#0 1 9,624 K ############################################################## iteration Completed : 1 ########## memory usage of perl start of execution for iter: 2 ####### perl.exe 18124 RDP-Tcp#0 1 9,640 K ############################################################## ########## memory usage of perl end of execution for iter: 2 ######## perl.exe 18124 RDP-Tcp#0 1 9,668 K ############################################################## iteration Completed : 2 [download] after 40 iterations memory has been increased as shown below: ########## memory usage of perl start of execution for iter: 45 ###### # perl.exe 18124 RDP-Tcp#0 1 11,308 K ############################################################## ########## memory usage of perl end of execution for iter: 45 ####### perl.exe 18124 RDP-Tcp#0 1 11,324 K ############################################################## iteration Completed : 45 ########## memory usage of perl start of execution for iter: 46 ###### # perl.exe 18124 RDP-Tcp#0 1 11,332 K ############################################################## ########## memory usage of perl end of execution for iter: 46 ####### perl.exe 18124 RDP-Tcp#0 1 11,348 K ############################################################## iteration Completed : 46 ########## memory usage of perl start of execution for iter: 47 ###### # perl.exe 18124 RDP-Tcp#0 1 11,356 K ############################################################## ########## memory usage of perl end of execution for iter: 47 ####### perl.exe 18124 RDP-Tcp#0 1 11,368 K ############################################################## iteration Completed : 47 ########## memory usage of perl start of execution for iter: 48 ###### # perl.exe 18124 RDP-Tcp#0 1 11,376 K ############################################################## ########## memory usage of perl end of execution for iter: 48 ####### perl.exe 18124 RDP-Tcp#0 1 11,392 K ############################################################## iteration Completed : 48 ########## memory usage of perl start of execution for iter: 49 ###### # perl.exe 18124 RDP-Tcp#0 1 11,404 K ############################################################## ########## memory usage of perl end of execution for iter: 49 ####### perl.exe 18124 RDP-Tcp#0 1 11,504 K ############################################################## iteration Completed : 49 ########## memory usage of perl end of execution ########### perl.exe 18124 RDP-Tcp#0 1 11,512 K ############################################################## [download] BTW as mentioned by Mr.Muskrat, I tried the script on linux station and the issue is not seen.	[reply] [d/l] [select]
Re^2: Memory Usage of Perl System command by hotchiwawa (Scribe) on Jan 10, 2016 at 13:51 UTC
Hi pperi2 :) The memory is maybe increasing because the memory of each shell execution has no time to be completely released that another shell command is coming, increasing also the number of process. What it can be useful is to get just the number of running process. Peace	[reply]
Re: Memory Usage of Perl System command by pperi2 (Novice) on Jan 11, 2016 at 23:01 UTC
Apparently, that looks like the bug in that version of perl as mentioned in this link: http://docs.activestate.com/activeperl/5.18/lib/pods/perldelta.html which got fixed from 5.18.4 and later versions.	[reply]