nandeya has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

Recently I began actively experimenting with mod_perl and one thing that I am struggling with a bit is the development process itself.

For example I'm cruising along making use of the Apache Request Object, using the param() method in multiple scripts which are then calling a local module. And I as I am developing that local module or one of the scripts and makinge a small change to the code, and then run it - oops, not quite how I want it, change this here, change that there, run it, whoops derferenced from that nested hash incorrectly, or whatever (details aren't as important - the general jist being run, small change, run, change, etc.).

The problem that I am running into given my understanding is that mod_perl has placed my scripts/modules in memory; and I don't have a firm grasp on how my changes to the scripts/local modules are being used (or even if they are due to the 'in-memory' script/module still being used, not what I just changed a few seconds before test-running it).

Any pointers on how I might go about addressing this, or if I am off my rocker (wouldn't be the first time or anywhere near the last). I would prefer to not have to stop and re-start the Apache server each time I am making a change (if that is one of the potential solutions). Is there some type of "force clear this script or module out of memory" type of step(s) you can take during the development process?

Thanks for any inputs,
nandeya

Replies are listed 'Best First'.
(jeffa) Re: mod_perl devlopment process
by jeffa (Bishop) on Jan 18, 2003 at 23:54 UTC
    maverick has been working on a "solution" for this for quite some time now. By "solution" i mean that the ability to make changes to a mod_perl module and see those changes without having to restart the webserver is just one feature of what he is building. The project is called SiteVoodoo and as soon as he, myself, and msemich can find the time we will package the whole kit and kaboodle up (possibly for CPAN and/or Freshmeat). Unfortunately for the public, each of us already knows how to create powerful websites with SiteVoodoo, so we have no flame under our seats to motivate us to make it user-friendly. ;)

    In the meantime, you can get a tar ball of the source code here. (And here is an example site that runs on SiteVoodoo. Note that it uses only a few of SiteVoodoo's features.) Hopefully maverick will come along and help explain how module reloading works, but i can tell you that it is no trivial matter. First, the notion of be able do such in a production environment is asking for trouble (performance-wise and possibly security-wise), so Voodoo defines a 'devel' mode (via a configuration file located in Apache's conf dif) which allows immediate changes in the code to show up without restarting the web server. Each of us (mav, semich, and myself) tends to work with at least two machines, one for development and one for production. The production machine has a different config file than the development machine, but the code base for the two machines is mostly identical (the difference being any new changes or bug fixes on the devel box).

    That's really about all that i can tell you about how module reloading works in Voodoo (and that was not a lot), but i can show you how to install Voodoo, and i assure you, it's really cool stuff! I did grep for Apache::Reload in Voodoo's source and found no matches ... methinks maverick might have re-invented a wheel. ;)

    Oh yeah, almost forgot ... the mod_perl Developer's Cookbook has been an invaluable addition to my library. The relevent recipe is 8.1, Recognizing Handler Changes, and the recommended solution is to use Apache::Reload instead of tampering directly with %INC.

    jeffa

    Random Voodoo Features:
    • auto reloading modules (in devel mode)
    • clean seperation of code and markup (uses HTML::Template)
    • manages user sessions (uses Apache::Session)
    • easy to write authentication/authorization checks
    • view debug info in browser (in devel mode)
    • provides persistant database connection (via Apache::DBI)
    • unique database abstraction - Voodoo writes code to add,update,delete records from your tables
    • built in record searching and pagination for the results
    • themeable
    • 100% mod_perl
Re: mod_perl devlopment process
by chromatic (Archbishop) on Jan 18, 2003 at 22:53 UTC

    Apache::Reload looks like it will do what you want.

    If I were in a similar situation, though, I wouldn't use it for two reasons. First, I prefer to catch my mistakes by writing tests before I write the code. Second, because I'd probably use the fake Apache module on the CPAN. (I can't remember its name right now.) I've developed by stopping and restarting Apache and mod_perl, and it bores me. I want faster feedback. :)

Re: mod_perl devlopment process
by perrin (Chancellor) on Jan 19, 2003 at 00:18 UTC
    You can use Apache::Reload or Apache::StatINC, as already pointed out. However, I would suggest you just get over it and restart the server. It's so simple and fast, and it leaves no possibility for problems. I prefer it to the attempted magic approaches.

    Incidentally, two expensive Java application servers that I have worked with require restarts to safely pick up changes in class files, and they take about 10 minutes to restart. You don't know how good you've got it until you sample the other side.

Re: mod_perl devlopment process
by nandeya (Monk) on Jan 19, 2003 at 04:32 UTC
    Thanks for inputs folks - much appreciated. How could I have not stumbled across the Apache::Reload module (guess it didn't bother me enough to really look into very well)... Thanks for pointing out the example in the mod_perl Developer's Cookbook as well jeffa, just picked it up a bit ago and should have noted the recipe/part you mention right at the start of Chapter 8.

    When I am changing local modules it is on a seperate devel machine so not a big deal for stop/starting Apache if doing that way (and incidently I would go bezerk if anything took 10 minutes to reboot - no wait, the Windows 2000 installed laptop I was given at work comes in around 4 1/2 minutes, good thing I partitioned off half the drive and installed Linux on and mostly use that).

    Was just thinking that there had to be another way (or ways) out there, and now I know. Thanks again for the inputs.

    nandeya
Re: mod_perl devlopment process
by CountZero (Bishop) on Jan 18, 2003 at 23:42 UTC

    An interesting thought. I have been doing exactly the same and did not run into problems with Apache not refreshing my scripts.

    Mind you, I was not using some local module but doing the changes to the CGI scripts themselves. I always thought (hoped?) that Apache would notice that the scipt was newer than the version held in memory and would do the "right thing".

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Re: mod_perl devlopment process
by zengargoyle (Deacon) on Jan 19, 2003 at 06:22 UTC
    $ kill -USR1 `cat /path/to/httpd.pid`

    after you've made changes to modules will cause apache to finish serving requests in progress and reload (rereads config files also). or use StatINC as others have mentioned to have it done automagically

    my understanding is that changes to scripts served up like plain old CGI will be noticed automatically, the server does the normal stat of the file and reload it if it's been modified. for stuff modules StatINC forces the same behavior.

Layering...
by pdcawley (Hermit) on Jan 19, 2003 at 22:52 UTC
    One option is to be rigourous about separating your application logic from the view related stuff. What this means is writing a layer between the mod_perl data structures and your internal objects, then you write a thin layer which converts from an Apache::Request to your internal Request structure and pass that inwards. The application should then return a datastructure/object/lump of XML that you render to HTML using the templating system of your choice.

    That way you get to test your application by feeding in appropriate internal Request structures and comparing the returned data structure with what you expect (and, of course, you should also write tests to ensure that Apache::Requests get converted correctly and that your data structures are rendered properl, but these sections can be tested independently of the application itself and can probably (in large part) be tested without needing to run Apache. Only when you've got your automated tests passing will you need to restart Apache (or use Apache::Reload) and do the eyeball test.

    You do have automated tests don't you?