By the way, the design of putting templates in the database is pretty bad for usability. Templates are files, and they should live on the filesystem. Otherwise, you end up with a big maintenance problem.

By the way, the design of putting templates in the database is pretty bad for usability.

Well, it is Slash we're talking about here :) Their design isn't perhaps as bad as it seems. The templates live on the filesystem, but after editing you run a tool to compile them, as Perl, into the database. They do this for speed. I haven't benchmarked it, so can't say if it works.

What really sucks about this is that each Apache/mod_perl child process caches all the templates, so if you make a minor change to a template you have to bounce Apache, or at least kill all its children.

Reading a file on a local filesystem is faster than reading from a database. Maybe they did it that way so they could share code between machines, but that's better done with NFS or rsync in my opinion. The compile step sounds annoying, as does having to restart when changes happen.

When templates live on the filesystem, changes can be automatically picked up by the running system just by stat'ing the file. That could be turned into a database check in their custom Provider class, but I suspect they did this to keep shared memory high. In my experience though, compiling the templates before forking didn't really decrease the amount of unshared memory.

Every person is going to make their own decision, but the amount of information and research each person employs in making that decision varies.

Saying that, I'd also consider additional factors like -

• is the server multi-homed, and does this affect issues in laying out sites
• does one layout or another help or hinder issues like deployment, redeployment, testing, etc
• does a chosen layout help segregate things that should be segregated, and collate things that should be collated

I admit, these issues are 'smaller' than what you have already raised, but they are important (well, to me at least)

I dont have any real answers, I tend to build web apps, not 'traditional' web sites, so my issues, I feel, are different to yours.

Also, I know I dont have good deployment policies, so I wont share them here ;-)

• images
• static PDFs (for help docs, privacy policy, and the like)
• stub CGI scripts to get CGI::Application up and running

Everything else, and I mean EVERYTHING else, is somewhere else. Often, different places. My perl modules are in /usr/local/lib. My templates and config files are in /apps/<APP>, and this includes my httpd.conf. /usr/local/apache2/conf/httpd.conf solely contains an Include directive. (I do it this way because I generally have 2-5 apps running on a given group of round-robin servers. My dev assignments don't often match my production assignments.)

I want to need to have root permissions to upgrade stuff on the production box. I want to know that only specific people can touch it, and every one of them (except me) doesn't. But, I'm also the admin for every bit of web prescence my employer has, so being paranoid is good job security. :-)

Oh - databases are for just that - data. Not only should templates not live in a database, but you shouldn't have any triggers, procedures, or the like that are application-specific, for several reasons.

• You sure your database is going to be used for that application and only that application?
• You sure you trust your DBA to be a programmer?
• You sure you trust your programmer to be a DBA?
• You sure you trust Oracle's patches? (Skip 9.2.0.3 - Oracle has said "Sorry bout your luck - hope you can save your data.")

If everything but your data is outside your database, you have a chance of changing vendors, especially to like PostgreSQL or MySQL. But, if you tie yourself to Sybase or MS SQL or Oracle by using their flavor of triggers, PL/SQL, etc ... your boss might not like to hear "I need another $XXX cause they're discontinuing our license." when there's a perfectly good open-source option.

Now, I'm not discounting triggers, procedures, or the like. But, IMHO, you should only use them for data-specific purposes. For example, using a trigger to do automatic history on a given table - that's a data question and properly belongs in the database. Using a trigger to make sure that X gets populated when Y does because it's easier than fixing the 9 places you populate Y ... that's not so good.

For the most part I agree that databases are (duh) for data. But Slash is a unique system. Templates live on the file system until they're compiled into the database as native Perl. Then they're read at server startup by mod_perl and cached in memory for the life of the server.

The drawbacks are numerous and obvious. The benefits of this I can see are:
• Faster web server startup than if the templates were on a filesystem, uncompiled.
• Single point of optimization (DB server) to improve template access speed.
• Fastest possible page rendering.
But maybe I'm missing something, and I certainly don't know everything about Slash. Maybe chromatic will reply.

Actually, I said /etc, but thats a minor quibble. Template files are configuration files. Configuration files belong in /etc. Where '/' is really located is the more interesting question.

There are 3 primary web application layouts, and a flexible application can (and should) support them all. You've got the single-instance-on-a-single-OS layout. You've got the mass vhosting, or multiple-instances-on-a-single-OS layout. And you've got a no-privilege or per-user layout. Fundamentally the only thing thats different between these layouts is where the root of their filesystem is. In mass vhosting scenarious I've used / var/www/<host>, I've used /home/<host>, you get the idea--and the configuration files for that instance have always gone in <prefix>/etc/ <application> because thats where configuration files go, in etc. Individual users hosted off a single web server really isn't any different from multiple vhosts hosted off a single web server, except frequently the former could have more privilege (additional unix accounts) at it disposal than the latter.

The ownership of configuration files will vary between applications, but the simple principle to work by is that whomever is responsible for maintaining the application should have the ability to write to the files, and applciation process owner needs to be able to read them.

For example, lets look at two layouts of the same application on a Debian server.

Given: Application "foo", in a single- instance-on-a-single-OS layout, where foo needs two configuration files "database.cfg" and "lookyfeely.cfg". Web server process runs as "www-data:www-data". The maintainer is "bob:bob". database.cfg has sensitive info in it.

$ ls -al /etc/foo
total 6
drwxr-xr-x   3 root   root     1024 2004- 02-09 15:21 .
drwxr-xr-x  66 root   root     4096 2004- 02-09 15:21 ..
-rw-r-----   1 bob    www-data    24 2000-05-28 06:59 database.cfg
-rw-r--r--   1 bob    bob      122 2004- 01-05 23:13 lookyfeely.cfg
[download]

Thats not a bad solution, the web server process can read database.cfg, bob can edit both files, and lookyfeely is readable by everybody, it contains no sensitive information so hiding its contents from the other system users is unneccesary and www-data needs to be able to read it anyway. This requires super- user privileges to configure initially, but thats OK on a single- instance-on-a-single-OS, thats how it should be.

Now lets look at, moving to a per-user layout.

Given: Application "foo", in a per-user layout, where foo needs two configuration files "database.cfg" and "lookyfeely.cfg". Web server process runs as "www-data:www-data". The maintainer is "alice:alice". database.cfg has sensitive info in it.

$ ls -alR /home/alice
/home/alice:
total 13
drwxr-xr-x   3 alice   alice     1024 2004- 02-09 15:21 .
drwxr-xr-x   6 root   root     4096 2004- 02-09 15:21 ..
drwxr-xr-x   6 alice   alice     1024 2004- 02-09 15:21 etc
drwxr-xr-x   3 alice   alice     1024 2004- 02-09 15:21 public_html

/home/alice/etc:
total 6
drwxr-xr-x   3 alice   alice     1024 2004- 02-09 15:21 .
drwxr-xr-x   6 root   root     4096 2004- 02-09 15:21 ..
-rw-r--r--   1 alice   alice      24 2000- 05-28 06:59 database.cfg
-rw-r--r--   1 alice   alice     122 2004- 01-05 23:13 lookyfeely.cfg

/home/alice/public_html:
...
[download]

Now we run into a problem because any other system user can read alice's database configuration file, and thats bad if the other system users aren't trusted. But there isn't let the web server read the configuration it needs to if alice doesn't have any additional privileges. This leaves you with a lot of options which I'm not going to delve deeply into except to say: more groups, setfacl, Apache's suexec, or any number of other hacks will work around this issue. Its not the applications job to dictate which method should be chosen, but if possible it shouldn't stand in the way of any of them.

Back to the meat of original issue, the web server does not ever need to serve configuration files. Ever. Doing so is adding needless complexity to a fragile system. So allow me to refute your 3 points on when its a good idea, because I assert it is never a good idea:

1) The application is, or instances thereof are, ment to be installed by users with no special privileges.

Then either the application must never store sensitive information, or the web server must be configured to use a tool like suexec so the CGI scripts run as that user. Either way, as templates normally don't contain sensitive information this is moot, and its still not a good reason to leave them in a publishable directory, the CGI application can just as easily specify an include path and pull the templates from a directory in the users control that isn't publishable by the web server.

2) The overhead for a different solution (e.g. precompiled templates in a database) isn't worth it.

meh--the filesystem is a database, you wouldn't put data that had nothing in common into the same table of a RDBMS, why do it in a filesystem?
-or-
meh--the filesystem is a namespace, you wouldn't put methods that had nothing in common in the same API, why do it in a filesystem?

3) The application is designed to run multiple instances, each potentially with different templates.

Every instance should have its own configuration directory. The reality is that when this is the case there's a very good chance that you're going to need configuration data for other applications belonging to the instance as well, best plan for it ahead of time.

Edited by Chady -- fixed formatting.

Thanks for the reply. Frankly I like the idea, and I always try to build things so people can store templates wherever they want. I just don't think it's always realistic or useful to require it.

I'm tired and don't want to belabor the issue; we can talk about it more tomorrow, 'cause I know you're way more experienced than I am.

I would say templates not intended to be served directly by the webserver are in a similar class to cgi scripts, and should be treated in a similar way.

For my work application we do indeed have multiple installations of the same application serving multiple virtual hosts. We have all the files needed for a virtual host residing under one directory hierarchy, but the docroot is just one of those:

web1% ls /www/www.somehostname.com
cgi-bin/  docs/  httpd.conf  lib/  logs/  template/
web1% 

Note that since different hosts may be running at different versions of the software, the perl modules are also installed locally (.../lib) rather than into any shared space.

The installer knows where each of these files should go and what permissions and ownership they should have, so we only have to get that right in one place.

Hugo