in reply to Re: Perl Script Causing high CPU usage
in thread Perl Script Causing high CPU usage

Hi All, Thanks a lot for so many ideas. To be more clear, in a single day system generates 5-6 lakh files in just 3-4 directories consuming around 10-14 GB space per day (individual file size is not huge - 12345 bytes, 5-6 digits number in bytes). There are files of 4-5 different patterns but count is huge. No date timestamp in file name (e.g ABC_YYYMMDD.log), just random 8 digits number in file names (i.e. ABC_????????.log) As per my observation, glob is not consuming high CPU. As soon as we are entering while loop, then can see CPU spikes. After commenting out mv, zip, unlink, just keeping print statements, I noticed CPU around 80%. After adding below in "next" section, CPU is under control 10-12% or even low to 4% sometime. select (undef, undef, undef, 0.250); but already process running slow because of huge number of files and stat calls as highlighted by you "all" above, cant afford sleep. Once again thanks to all of you, will try out various ideas given by you. Thanks for letting me know about File::Find::Rule as well.

  • Comment on Re^2: Perl Script Causing high CPU usage

Replies are listed 'Best First'.
Re^3: Perl Script Causing high CPU usage
by marinersk (Priest) on Oct 01, 2013 at 13:55 UTC
    Good luck in the hunt -- if the loop is using lots of CPU just to do print statements, then I don't know that there's much you can do to limit its CPU utilization. Sounds like the OS is giving your script everything it needs; are you sure it's a problem?

    As a complete side note, I am deadly curious -- what is a lakh file? Is that some kind of special log format or something?

      I suspect that the poster is using the Indian term for 100,000


      Information about American English usage here and here. Floating point issues? Please read this before posting. — emc