caedes has asked for the wisdom of the Perl Monks concerning the following question:

I'm working on a simple little caching mechanism for my webapp when I move it over to mod_perl. I've looked at some of the CPAN solutions, but I really need to do this from scratch since this app. will have to run on systems where new modules can't/won't be installed.

Basically what I have is a subroutine Cache::cache() which you call with three arguments: a key, a subroutine reference, and a timeout value. The cache subroutine decides whether or not to run the subref and returns the output of the subref (either the cached or new output).

What do the wise monks think of this approach? 

#!/usr/bin/perl -w
# This script runs under mod_perl

use strict;
use CGI qw(:standard);

my %data = Cache::cache('info',\&do_it,200);

print header('text/html');
print start_html();
print p([$data{data},$data{source},$data{'time'},"Process: $$"]);
print end_html();

sub do_it {
    # do something that takes a long time 
    my $text = `ls -lR /`;
    return length($text);
    }

package Cache;

sub cache {
    our (%time, %asdf);
    my ($key, $sub, $time) = @_;
    if((not defined $asdf{$key}) or (not defined $time{$key}) or ($tim
+e{$key} < time - $time)){
        $time{$key} = time;
        $asdf{$key} = &$sub;
        my %out = (data => $asdf{$key}, 'time' => $time{$key}, source 
+=> 'db_query');
        return %out;
        }
    else{
        my %out = (data => $asdf{$key}, 'time' => $time{$key}, source 
+=> 'cache_query');
        return %out;
        }
    }

[download]

-caedes

Replies are listed 'Best First'.
Re: Caching data with mod_perl
by Revelation (Deacon) on Jul 14, 2002 at 02:38 UTC
    A couple things:
    1. What is the specific purpose of this Caching?
    2. Why are you using this aproach?
    My personal opinion is that Dominus' memoize module is *perfect* for this purpose, and if you're able to install this module, there should be no problem with installing Memoize (even without cpan)... You would only have to code a time check into a module using Memoize, or Memoize itself.

    However, if you can't use Memoize, please answer my questions! :) I ask them because one may propose a differant module, or a differant tact, depending on how you propose to do this. Of hand, if I didn't use a module, I would create subroutines, so that I wouldn't have to reload subroutines subrefs from a hash all the time: 
    sub CheckCache
{
   my $self     = shift;
   my $sub      = shift;
   my $subref   = shift;
   my $time     = shift;
   if (not defined($subtimes{$sub}) or $subtimes{$sub}) < time - $time
+) { $self->newsub($subref,$sub,$time);} else {goto &$sub;}
 
}

sub newsub
{
    my $self  = shift;
    ref $self or die("$self is not an object");
    my $subref   = shift;
    my $sub      = shift;
    my $time     = shift;
    $subtimes{$sub} = $time;
    *$sub = &$subref;
    
    # no reason to play this game every time
    goto &$sub;
}

    [download]
    And use it by doing 
    my $cache = new Cache::Cache;
$cache->CheckCache("key",\&sub,$time);  # Object oriented

Or

Cache::CheckCache("key",\&sub,$time); # in non OO

    [download]
    Note: For non OO, $self should not be included, and lines with $self are probably not necessary. Personally, I would use OO for this task, because I think it would be easier to expand, however it is *your choice*.

    Update To Response: This is the exact purpose of Memoize, however I have given you what is an alternative way of doing this in the above code. The module I created is much like your own, just a module: You can learn from other's modules as well as your own, this is one way to do this, although I prefer memoize.

    On the subject of using 'our' in the module, I was under the impression that our went out of scope. use vars() allows for the hash to remain in the httpd process, so that it doesn't have to be reloaded, if the script is called by two differant people in the same time period. This means that Joe and Larry can use the cached results of the same routine (I only cached the routine, but you can probably figure out how to cache results...) Although the best option would be to cache results, and if the time period had passed, to just re-execute the ready-made subroutine.

    In addition, the reason I was using subroutines is that retrieving a subref and calling it from a hash is *probably* more expensive than just executing a created subroutine. Just think of my post as an alternate aproach :)
[reply]
[d/l]
[select]

      You really should have a look at Apache::Cache, too! It's doing exactly what you described.. The bad news is, that it doesn't come with the standard package and is likely to not be installed on the machine you want to have it run on.

      A more general approach would be using Apache::Storage, which isn't included by default either :( Both of the above modules provide inter-process shared memory, btw.

      If you really do want to do it yourself, I would at least look how this modules are doing their job; you probably can learn a lot by doing this.. Especially using shared memory is something you really do want to have..

      Regards,
      -octo

[reply]
        In general, you get better performance from disk-based solutions to sharing. I'm doing some benchmarking right now, and so far MLDBM::Sync (using SDBM_File) and Cache::Mmap are beating the pants off modules that use shared memory. The only exception to this is IPC::MM which has phenomenal performance.
[reply]
      The purpose is to keep the "do_it()" and other subroutines (which take a long time) from running every time.
      I don't really know what you mean by my "approach", this is just what I came up with to cache data with mod_perl. Mostly what I'm looking for is some guidance on the pros and cons of doing it the way I've shown: such as using "our" in the Cache module.

      As I said before, I'm only interested in learning about caching information within a mod_perl proccess. I'm not interested in using a module to do it because then I don't learn anything. :-)

      -caedes

[reply]

        One con of is that each child process will have to maintain its own cache -- written memory will be copied, not shared, between processes.

        I'm not interested in using a module to do it because then I don't learn anything.

        You are allowed to read the source code of the Memoize module, as well as MJD's article on the subject. Seriously, you're more likely to learn a hundred wrong ways to do something than you are to stumble across a good approach by accident.

[reply]

        I agree with chromatic++ on this one. Avoiding modules in order to "learn" something is an idea that took me a long time to overcome, and I still wrestle with it quite frequently. Many problems seem deceptively simple, but you are likely to reinvent nothing other than the same mediocre (to avoid a worse term) solution with the same inherent problems that many other novices before you came up with. (Manually parsing CGI parameters or recursing through directories f.ex are commonly undertaken problems that you actually need a very solid understanding of the subject at hand for before you can take a stab at them yourself.)

        I don't mean to sound berating. I started out as a total newbie myself, as has everyone else. I've written bad code in the past that I now sneer at, and I still continue to learn how to write things in better ways every day. By all means do play around; I don't wish to discourage you from experimenting, and learning is best done by making your own mistakes and taking lessons home from them. Growth comes through error. You are definitely encouraged, as chromatic noted, to study the modules' source code and learn from it.

        However, when writing production code, please stick to modules. It means you will have less code to write yourself and consequently to maintain. And when you look back at the source in 2 years, or your successor does then, there will be less things to figure out. Also, the modules will correctly handle a lot of details that you likely didn't even know about at the time. It is no shame or stigma that your experience is as of yet limited but please acknowledge and remember it when writing code that others need to rely upon. Priorities differ then. It was a hard lesson that took me a lot of time to learn.

        Makeshifts last the longest.

[reply]
      Using 'our' as I have done results in the data being cached correctly, so I certain that it doesn't go out of scope.

      -caedes

[reply]
Re: Caching data with mod_perl
by IlyaM (Parson) on Jul 14, 2002 at 18:41 UTC
    I've looked at some of the CPAN solutions, but I really need to do this from scratch since this app. will have to run on systems where new modules can't/won't be installed.

    Just bundle all required CPAN modules with your app. Now when I've just solved your first problem :) you can revisit CPAN again. In addition to already mentioned modules I suggest to take a look on Cache::Cache.

    --
    Ilya Martynov (http://martynov.org/)

[reply]

Back to Seekers of Perl Wisdom