tc1364 has asked for the wisdom of the Perl Monks concerning the following question:

Can you please assist me with resolving a trouble with some code that at times is eating up allot of CPU resouces? The code is driven by Perl and calls Expect. The Expect code is basically dumping numerous tables to standard out that is captured into one file. A few of these table dumps are very large and begin eating up allot of CPU resources. The basic flow is the Perl code starts on a Midwest server, makes a system call to Expect, Expect logs into a West server and begins dumping table information to standard out that is being recorded in one file on the Midwest server where the Perl code was launched. 
    eval {
        alarm(9000);
        $expect_flg = $expect_flg + 1;
        $file = "$workdir/$clli.log";
        system($pb_dms, $datakit, $workdir, $file);
        alarm(0);
        $expect_flg = 0;
    };

[download]

Edit by castaway - repaired code tags

Replies are listed 'Best First'.
Re: Resource Hog
by jbrugger (Parson) on Apr 06, 2005 at 21:03 UTC
    seeing your code, i assume you're on a unix like machine. Try to set the nice level of your program. If it's the only running process, it still takes a lot of cpu-time, but if another program with a higher nice level gets in, it reduces it's load.

    Since you use system commands, you could run them like 
    ...

if ($nicelevel ne "") {
  $cmd = "nice -$nicelevel " . $cmd;
}
system $cmd;

    [download]

    Or You could look here

    "We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise." - Larry Wall.
[reply]
[d/l]
      Yes, the code is running on a Unix machine and the nice level is 1 when observing the performance using the "top" command. The other code running on the server has a nice level of zero. Maybe I am reading the "top" command wrong but under the CPU column, the percentage reaches 40 percent at times for these large table dumps with a CPU time reaching 6 minutes. There is always some other processes running on this server.
[reply]
Re: Resource Hog
by starbolin (Hermit) on Apr 07, 2005 at 00:42 UTC

    I'm not sure what your question is. Do you want to modify the code or are trying to modify the system so the the existing code has less impact? What behavior are you looking for?

[reply]
Re: Resource Hog
by eXile (Priest) on Apr 07, 2005 at 02:42 UTC
    sometimes the most cost-effective way to solve CPU-related problems is to buy a faster CPU.
[reply]
      Thank you for support regarding this question! One other question regarding this, do you think it would help solve this trouble to have the Expect code written in Perl? The current Expect code is the original stuff and is not the Perl version.
[reply]

        Are you talking about Don Libes' Expect written in Tcl? The picture I'm begining to get is of a Perl script calling a Tcl script which causes a dump to STDOUT. Is this correct? The problem then would be in the character based handling of a file instead of a protocol more appropriate for transfering large files over high latency links. The language the scripting is written in is not an issue. The first thing I would do would be to rewrite the Expect script to use FTP instead of a dump to STDOUT.

        Playing with nice and such won't do you any good as much of the time is being spent in doing I/O and the OS sees the activity in STDIO and overides the priority.

        s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}
[reply]
Re: Resource Hog
by starbolin (Hermit) on Apr 07, 2005 at 17:06 UTC

    If your application is I/O bound, which from your description it appears to be, then nice won't do much for you. The OS is already handing cycles to other processes while waiting for I/O opperations. Although the high CPU utilization would point to some serious CPU cycles being spent somewhere. I can't tell from your description where that would be. A better indication of system load can be had with the uptime command.

    You don't say what unix you're running on. It makes a difference as System V multitasks different than BSD.

    s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}
[reply]
      Sun Microsystems Inc. SunOS 5.8 Generic February 2000 Sun Ultra 60
[reply]

        SunOS is BSD based. BSD was written to see the activity on STDIO and modifiy the job priorities accordingly. I can't speak to what tweeks SUN has made.

        s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}
[reply]

Back to Seekers of Perl Wisdom