SOLVED Updated to perl 5.20.2 and all problems vanished.

Hello! Been noticing some strange memory issues with a script I'm working on, and have come up with this to highlight the issues

use strict; use warnings; use threads; use threads::shared; $|++; my $DONE :shared = 0; my $lock :shared; my $execMethod = $ARGV[0] || 0; if($execMethod !~ /[12345]/){ print "Must pass an exec method:\n"; print "1 = backticks\n"; print "2 = backticks synchronized\n"; print "3 = open\n"; print "4 = open synchronized\n"; print "5 = system\n"; exit 1; } sub execute{ my $cmd = shift; if($execMethod == 1){ `$cmd` }elsif($execMethod == 2){ lock $lock; `$cmd` }elsif($execMethod == 3){ open(my $fs, "-|", $cmd); foreach(<$fs>){}; close $fs; }elsif($execMethod == 4){ lock $lock; open(my $fs, "-|", $cmd); foreach(<$fs>){}; close $fs; }elsif($execMethod == 5){ system($cmd . ">nul"); } } sub worker{ while(!$DONE){ execute('echo hello world'); } } my @workers = map threads->create( \&worker), (1..30); print "Press <enter> to terminate\n"; <STDIN>; $DONE = 1; $_->join() for @workers;

Script is invoked with <script>.pl executionMethod

Question 1 relates to memory usage. Methods 2, 4, and 5 all exhibit a permanent memory creep. Is there a way to fix this and let this script run forever? If the solution is periodically join threads, this is not feasible. I have tried this a couple years ago and came across some bug but outlined in this node and changing to an approach like this would require far too much testing to ensure no other bugs occur. If there really is no solution, would be good to know :)

Question 2: For methods 1 and 3, why do i need to synchronize them? If I do not, the script just hangs up usually after only a few seconds. Synchronizing this access kind of seems to defeat the purpose of running system commands in threads, is there something I can do differently?

*I have been running this on Windows 2008 R2, using perl 5.18.2

Thanks in advance for all your help

UPDATE 1: Here is the log file from perfmon running test 5 for a little over a minute and eating roughly 150MB.

UPDATE 2:

perl -v output

This is perl 5, version 18, subversion 1 (v5.18.1) built for MSWin32-x +64-multi-thread (with 1 registered patch, see perl -V for more detail) Copyright 1987-2013, Larry Wall Binary build 1800 [297570] provided by ActiveState http://www.ActiveSt +ate.com Built Sep 20 2013 15:07:17 Perl may be copied only under the terms of either the Artistic License + or the GNU General Public License, which may be found in the Perl 5 source ki +t. Complete documentation for Perl, including FAQ lists, should be found +on this system using "man perl" or "perldoc perl". If you have access to + the Internet, point your browser at http://www.perl.org/, the Perl Home Pa +ge.

systeminfo

Host Name: VCLOUD291 OS Name: Microsoft Windows Server 2008 R2 Enterprise OS Version: 6.1.7601 Service Pack 1 Build 7601 OS Manufacturer: Microsoft Corporation OS Configuration: Standalone Server OS Build Type: Multiprocessor Free Registered Owner: Windows User Registered Organization: Product ID: 55041-507-5915375-84291 Original Install Date: 4/23/2014, 12:10:03 PM System Boot Time: 7/31/2015, 1:49:06 PM System Manufacturer: Microsoft Corporation System Model: Virtual Machine System Type: x64-based PC Processor(s): 1 Processor(s) Installed. [01]: Intel64 Family 6 Model 46 Stepping 6 +GenuineIntel ~1862 M hz BIOS Version: American Megatrends Inc. 090004 , 3/19/2009 Windows Directory: C:\Windows System Directory: C:\Windows\system32 Boot Device: \Device\HarddiskVolume1 System Locale: en-us;English (United States) Input Locale: en-us;English (United States) Time Zone: (UTC-07:00) Arizona Total Physical Memory: 4,096 MB Available Physical Memory: 1,635 MB Virtual Memory: Max Size: 6,143 MB Virtual Memory: Available: 3,364 MB Virtual Memory: In Use: 2,779 MB Page File Location(s): C:\pagefile.sys Domain: WORKGROUP Logon Server: \\VCLOUD291 Hotfix(s): 46 Hotfix(s) Installed.

In reply to Perl system command memory usage in threads by rmahin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.