Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

pulling content from db - is it a good idea?

by kiat (Vicar)
on Nov 25, 2003 at 10:58 UTC ( #309853=perlquestion: print w/replies, xml ) Need Help??

kiat has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I intend to have an (or index.cgi) served as the default html page. Instead of hard-coding the page's contents into, I'm thinking of storing them in a database or in a module file and have retrieve the appropriate contents and display them onto a webpage. Is that a good idea?

If the contents reside in the database (this database is also for holding other information, such as members' profiles, presumably in a table of its own), persons who have access to an admin script can use it to change parts of the contents in the database, without having to touch

Another option I've thought of is to store the contents in a hash in a module file and have retrieve the contents from there.

Are those sensible things to do?

  • Comment on pulling content from db - is it a good idea?

Replies are listed 'Best First'.
Re: pulling content from db - is it a good idea?
by Roger (Parson) on Nov 25, 2003 at 11:31 UTC
    I would suggest not to store the HTML contents in a database. It just makes it inconvenient to view and edit the pages.

    I would rather make a distinction between logic data and physical layout. Build a two layer system:
  • A logical data layer that represents the data to be shown;
  • A physical presentation layer that builds the webpage;
    and separate the two layers as much as possible.

    For example, I would classify the member's information as logical data, and store them in database table(s). I would use modules like CGI, HTML::Template, etc to build the look & feel of the webpage - the physical layer.

    +-----------------+ | HTML TEMPLATE | presentation layer | | +-----------------+ | | HTML::Template +-------------+ | PERL SCRIPT |---------> build the final HTML page +-------------+ CGI | DBI, DBD | +-----------------+ | MEMBER's INFO | logical data layer +-----------------+ | DATABASE | +-----------------+
      hm, HTML::Template seems to be real popular here.
      did someone ever use Template Toolkit? it seems to me, that itīs mutch more purefull and versatile.
        Well, I mentioned HTML::Template just as an example, not to force the use of it. You can use any toolkit you like to build the HTML pages of course. I did say 'etc', didn't I?


      Thanks, Roger! I'll keep your proposal in mind :)

      One of my considerations is to make it easy to pull out the data for modifications by authorised persons.

      I was also thinking that if I had everything in a database, it might make managing every piece of data in the site easier to manage.

Re: pulling content from db - is it a good idea?
by moxliukas (Curate) on Nov 25, 2003 at 11:27 UTC

    Well, I have done this before. Putting content into a database is not a bad idea (and as a bonus you can easily search the content of your website because all the hard work is done via database engine (I am assuming relational database))

    Another alternative is to have a separate template directory where you put all contents (you fill the template using a module HTML::Template or any other templating solution). Templating clearly separates code from presentation so it is very handy when you need to change the overal appearance of the website

    I personally use Templates with all the content that changes from page to page stored in a database

Re: pulling content from db - is it a good idea?
by Abigail-II (Bishop) on Nov 25, 2003 at 12:08 UTC
    First of all, there isn't anything Perl related to this. You'd have the same issues if you had coded your website in C, Java or some other language.

    As for your question, whether it's a good idea or not depends (as usual). Some important factors are: how volatile is the data? How much do you care about performance? How much do the users care about performance?

    There is a third option that you haven't mentioned: store the (not marked up) content in a database. Extract the content, and create a static index.html page. Whenever the content in the database changes, update your index.html.


      The last option is particularly useful if the HTML page produced only changes once a day but the length of time that the script takes to run is long or consumes many resources.

      An example would be if the web page showed a daily status report derived from a number of processor intensive SQL queries. The same script (using CGI and HTML::Template) can be scheduled to produce a fresh static HTML page on a daily basis.


        I'd say that even if the content changes every five minutes, it's in most cases worthwhile to generate a static HTML page every five minutes. About the only time where that's a bad idea is if all of the following are true:
        • Generating the HTML page consumes a lot of resources.
        • You get less than one hit on the index page every five minutes.
        • The server is busy doing things other than serving the index page.


      Thanks, you gave me an idea.

      I could have the content in the database, and like you suggested, update a template page (e.g. main.tmpl) when the content changes. I need to have an as the default page because I figure I need to show some dynamic information on the page (e.g. members' online). I could then use HTML::Template within to serve out main.tmpl.

      What do yout think?

        Members online is some information that doesn't look like it depends on the request being done. Often, for web pages, "members online" isn't anymore more than "this many members accessed the site in the last X minutes". If X is say, 2 minutes, and your site is very active, you might consider updating your index page every 2 minutes (if your site isn't very active, updating every 2 minutes doesn't hurt).

        Unless it's about pages whose content isn't known until the request is made, I wouldn't dismiss static pages to quickly.


Re: pulling content from db - is it a good idea?
by bradcathey (Prior) on Nov 25, 2003 at 13:21 UTC
    I not sure whether you mean:

    • you want to store the content, and not the HTML in this database, or...
    • you want to store both in the database

    In either case (thought not so sure you'd want to try the latter), what you are proposing is nothing new. In fact, unless I'm not reading correctly between the lines, you are talking about a database-driven content management system (to make it sound important). This is the underlying structure for more and more Web sites, and is the future.

    Before joining the order of Perl Monks, I used to store all content in flat files, and HTML in the Perl scripts that would pull the content from the files and then render the HTML pages as "here documents."

    Well, those days are gloriously over. Thanks to HTML::Template and MySQL. The latter makes content storage and retrieval nice and tidy, and HTML::Template allows you to build and maintain standalone HTML pages with placeholders for the content to come later. Perl scripts pull content from the database and H::T renders the HTML page with the content inserted.

    Of course, you will need to supply a GUI for your users (can be done through something as simple as HTML forms or as complicated as the Microsoft DHTML Editing Component).

    I hope I got what you were asking, it's just that the practice is so common today that I thought I might have missed the point of your question. Apologies if I did, and write on if I didn't.

    "A little yeast leavens the whole dough."

      It's funny how everything goes around in circular fashion. When I started doing web development, servers didn't have much in the way of capacity, and CGIs were extremely expensive, so everything was static. Dynamic content was something reserved for a small section of the site, and generally served off a different machine (as I remember, we had some big 8-processor Sun boxes for this - heck, it was 1996).

      As time went on, the software improved, more features were desired, and dynamic content started to creep into more areas of out sites. Our front-line webservers started serving quasi-dynamic content, and those 8-processor boxes went by the wayside.

      Nowadays, we're almost back where we started. We have front-line webservers that do nothing except serve static HTML, though we do use mod_include SSI directives as well (something that could be done in perl). We generally don't do much in dynamic content, except for full-blown applications. And we serve a *lot* of content every day.

      With all that - nothing beats static HTML in terms of performance or server response time. And the more systems/applications you add to a page, the fewer pages you're able to serve, the more your response time goes up, and the more hardware you'll need to support your site.

      Things to think about:

      • Serve static HTML. If you've got dynamic components, how often do they need to update? Do they need to update on every request? If not, that's not dynamic - it's static that just needs to change often - the data can probably be written out to the filesystem and included in the page in any number of ways.
      • If you need dynamic content on every page - use HTML::Template, it's the fastest templating system for perl out there. And look into caching your templates, it will save a lot of memory in a server environment. Screw 'more powerful' templating systems - HTML::Template does anything you'll need, and it won't slow you down. I've never been in a situation where it hasn't done what I need.
      • Minimize the amount of content that needs to be re-written. If you're talking about sidebar content, don't re-write the main body at all! Save it elsewhere and slurp it in!
      • KEEP THE DB AWAY FROM THE FRONT LINE! Nothing slows a site down like hitting a database - even if you're caching your DB connections, you're talking about a whole other application that you're interacting with. Get a lot of traffic, and your DB will be holding you back.
      • If you absolutely need to hit a data store on every single page, what can you do to speed it up? Can you pre-load all the data from the DB before you start serving connections? Can you use a BerkleyDB?

      I don't want to discourage you from going into dynamic website construction - it's where things get interesting. But if you're not careful, you can find yourself with a server melting down under the load, and few quick remedies. Try a few things, see what it gets you, but keep the above in mind if people start to really like what you're doing. Good luck! :)

      Thanks, Brad!

      I was referring to storing just the content (minus the html) in the database.

      To reiterate your points: It's pretty the normal thing to do these days to store content in the database and have a perl script pull out the content and display it via HTML:Template.

      Is that right?

        Yup, common as dirt. There are several Tutorials here at the Monastery. Here's a taste of how it works:

        The HTML (has to be saved with .tmpl extension):
        <tr> <td><span class="header"> <!-- placeholder --> <tmpl_var name=listfor>'s List </span> </td> </tr> <tr> <td> <!-- placeholder --> <tmpl_var name="list"> </td> </tr>
        The Perl:
        use HTML::Template; #tell H-T what page to display $template = HTML::Template -> new(filename => "displaylists.tmpl"); + #connect to DB, SELECT, etc. then... @data = $sth->fetchrow_array(); #assign data to placeholders $template ->param(listfor => $data[1], list=> $data[2]); print "Content-type: text/html\n\n"; print $template->output; #display html
        It's pretty easy once you get the hang of it. And the HTML designers will appreciate not having their design packed away in some Perl script.

        "A little yeast leavens the whole dough."

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://309853]
Approved by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (1)
As of 2022-07-03 21:13 GMT
Find Nodes?
    Voting Booth?

    No recent polls found