geektron has asked for the wisdom of the Perl Monks concerning the following question:

i had to recently rewrite a mod_perl handler that was using Apache::CacheContent ( see: Apache::CacheContent and unforeseen side effects ).

now, i'm seeing way too much CPU utilization from the new Cache::FileCache-based handler. (at least, the bossman thinks it's too much load.)

with maybe a few requests/sec to this site, CPU is somewhere around 30% ... I've added PID logging in the handler to confirm that the CPU spikes are from this handler, and not from some other runaway process....

the output from top looks something like this:

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMM +AND 6199 nobody 15 0 14384 14M 5520 S 29.6 0.6 0:07 0 http +d 6251 nobody 15 0 14816 14M 5808 S 12.9 0.7 0:05 0 http +d
and the PIDs are verified as part of the handler, and one step further, the spikes are from cache access, not from cache setting.

is there any way to lighten the CPU load?

and the relevant part of the handler ...

$did ||= 1100; my $cache = new Cache::FileCache ( { namespace => 'mySpace', default_expires_in => '600', } ); my $data = $cache->get( $did ); warn "using data from cache! (PID: $$) " if DEBUG and $data; if ( ! $data ) { use Net::SSLeay qw(get_https post_https sslcat make_headers make_ +form); my $url = "*****"; $SIG{ALRM} = sub { return DECLINED; }; alarm(15); my ($page, $response, %reply_headers) = get_https($secureHost, 443, "$url"); alarm(0); ### process $page to get $data .... warn "setting cache now for $did (PID: $$) " if DEBUG; $cache->set( $did, $data, "600" ); }

Replies are listed 'Best First'.
Re: Cache::FileCache, CPU utilization, and bottlenecks ....
by perrin (Chancellor) on Jan 10, 2005 at 23:39 UTC
    I second the advice about profiling your handler. Keep in mind though, Cache::FileCache is relatively slow compared to some cache modules. See this page for some comparisons.
      Another option to consider would be Cache::Memcached; it's supposed to be quite fast.
        well, i've started working towards that today ...but i need to track down some socket errors, specifically:
        Bad arg length for Socket::pack_sockaddr_in, length is 0, should be 4 + at /usr/lib/perl5/5.8.0/i386-linux-thread-multi/Socket.pm line 373

        i found the problem. i wasn't passing the port on the IP address ... DOH!

      i'll check those benchmarks, and possibly shift to another, faster caching mechanism.

      peak times for this site hammers the server, and since it's not a dedicated one, i need to make sure it's as fast as can be so other domains don't start getting denied requests (from timeouts or whatever)

Re: Cache::FileCache, CPU utilization, and bottlenecks ....
by redhotpenguin (Deacon) on Jan 10, 2005 at 22:38 UTC
    One of the first things you should do is profile your handler and look at the results to see the resource distribution within your request.
      well, profiling is now on tomorrow's to-do list.
      well, tomorrow came early (because it seemed like an interesting thing to do), but i must be missing something.

      the perldoc indicates that all that's needed is an inclusion of this line:

      #in httpd.conf PerlModule Apache::DProf
      i've added that in the VirtualHost container, but i don't see *any* dprof directories, either in the server root for the VH, or in the log dirs .. <p< i've run a  find / -type d -name dprof -print and it turned up nothing ....

      so, does this need to be in the main httpd.conf, and i'm going to have to try and isolate the pids for one VH, or is there something i'm missing, like making the handler a subclass of Apache::DProf ?

        There's a chapter in the Practical mod_perl book by Stas Bekman which has an overview of profiling and includes a section on Apache::DProf , this might be able to help you out.

        From other posts on this thread though it might just be worth it to go with Cache::Memcached, which should provide a big speedup.