in reply to Performance Question

Granted I don't know why you have both high idle and low i/o wait, but my eye is attracted to this part of your post:

"...and prints the line back out to another file on the same disk."

Is there any way you can have the input and the output files on separate volumes (or strings)? You might have faith in the machine's RAID setup (assuming there is one), but it's not guaranteed to save your bacon. Give it a try, even if the input (or output) has to be in a different part of a LAN.

Replies are listed 'Best First'.
Re: Re: Performance Question
by mr_mischief (Monsignor) on May 08, 2002 at 21:09 UTC
    I'm not sure I'd consider 76% "high idle". As a matter of fact, if the one program in question is creating most of the 24% processor load and it's being run under a regular user account, it might be throttled for CPU utilization. Many Unix systems only only a regular user up to 10% or 15% of processor cycles.

    OTOH, you're definitely right to suggest using separate volumes. If the input file, output file, and swap partition are all on the same RAID logical disk or the same physical disk, it's probably a thrashing issue. I've seen a mail server increase performance by as much as 400% by simply moving the /var/log directory from a RAID volume to a separate disk. It hadn't been doing so well with logging, queueing, and spooling all on one array.

    If it's possible to run something like this as root and the machine loading down isn't an issue, then it's a good idea to run it as root. It's definitely a good idea to look at the disk subsystem, too. Perl solutions may help, too, but I'd look at system issues first in this case.

    Christopher E. Stith
    Do not try to debug the program, for that is impossible. Instead, try only to relaize the truth. There is no bug. Then you will find that it is not the program that bends, but only the list of features.
[reply]

In Section Seekers of Perl Wisdom