in reply to File::Find hogging memmory.

Hi monks,
Thanks a lot for the guidance!!!   I am here to provide more info regarding the issue.

The version of perl in the servers are
1) server where the process hogs more memmory - v5.8.0 built for PA-RISC2.0-thread-multi-LP64
2) The other server - v5.8.0 built for PA-RISC1.1-thread-multi

Both the servers are having the same OS HP-UX B.11.00. I have verified that the test script I run is the same on both machines, which is as follows:.

#!/usr/local/bin/perl use File::Find; sub findtime { if ( -f "$File::Find::name" && "$File::Find::name" =~ /.*\.txt/i) { } } while (true) { sleep 3; find(\&findtime,"/test"); } and the number of files( both total and *.txt files) under the directo +ry /test is the same. Additionally, i have observed that the memmory +hogged by the server(which has the issue), apart from being huge at t +he initiation itself, grows with time. I have done a diff between the File/Find.pm files of the two servers a +nd noticed that following are the differences.( lets call the server +hogging memmory as s1 and the other as s2) s1 -- 17c17 < find(\&wanted, @directories_to_seach); --- s2 --- > find(\&wanted, @directories_to_search); ---------------------------------------------------------------------- +--- s1 -- 42c42 < order they are given. In essense, it works from the top down. --- s2 -- > order they are given. In essence, it works from the top down. ---------------------------------------------------------------------- +---- s1 -- 571c571 < local($dir, $name, $fullname, $prune); --- s2 -- > local($dir, $name, $fullname, $prune, $_); ---------------------------------------------------------------------- +----- s2 -- 702a703 > $_ = $name if $no_chdir;

Apparently, the Find.pm in s1 looks like to have undergone some manipulations, by the root, which is highly unlikely. Please let me know if I can provide any further info, before reaching some conclusion.

janitored by ybiC: Removed frowned-upon <pre> tags, minor format tweaks for legibility

Replies are listed 'Best First'.
Re^2: File::Find hogging memmory.
by atcroft (Abbot) on Oct 13, 2004 at 16:53 UTC

    First of all, did you follow gellyfish's suggestion and compare the versions of the File::Find module itself? Did they report version differences?

    Secondly, while you included code to show how it is deciding to do something with the desired files, you showed nothing regarding what you are doing with the match. Are you storing something related to the match in an array, hash, object, or other "growable" data structure of some kind?

    My only other concern would be in that you are creating what is essentially a daemon process that runs continuously, with short (3s) pauses between processing. Not knowing your particular application, I cannot say this is or is not the best way to approach it, but if it is not essential to the application, you might consider limiting the number of times the app runs, in combination with the system's cron (or similar) feature, to keep the app from running too long if it does indeed have some form of memory leak.

    Not sure that helps, but hope it at least gives you something to look at.

      The two Find.pm versions are compared and they are found to be the same (1.4). Still when I do a manual diff between them I could see the differences, as mentionjed in the previous mail from me. Please not thet this Find opetations are not part of any application or any other processes. I have written some test scripts that have only one Find operation in it, and obserbved that one of the servers hoggs a lot of tmemmory for the process. Please share ur ideas...