qxotic has asked for the wisdom of the Perl Monks concerning the following question:

Sorry if I am repeating myself, I tried to post this once but didn't see it show up. What is the best method .. flatline data.. database.. or whatever for storing data that is displayed on your website but has to be updated on the fly based upon user input or selection? Thanks. Qxotic.

Replies are listed 'Best First'.
Re: Data on the fly?
by data64 (Chaplain) on Mar 01, 2003 at 04:16 UTC

    Short answer: It depends.

    Below is a short list of questions you might need to consider while making this decision.

    • How much data are you thinking about?
    • Can you give a better description of the data?
    • What kind of OS did you have in mind for your webserver and what webserver is it ?
    • How important is security of this data to you ?
    • How important is the integrity of the data to you, ie. is losing part of it because of a file corruption or something okay with you ?
    • Do you need to search through the data to locate a particular item ?
    • How many users do you estimate accessing the data ?
    • Are the web users also updating the data and could more than one user be updating the same item of data at the same time ?

    Just a tongue-tied, twisted, earth-bound misfit. -- Pink Floyd

      Data file size... a couple hundred rows, 5-6 columns. type .. inventory.. low inventory items, if one person orders and it's no longer avail it needs to be reflected quickly. Server.. have not chosen one yet.. still in planning stages I don't want to lose any data. File will be displayed in order.. but update as refreshed, inventory may change rapidly. searching.. yes, there would be some searching involved. no. of users... not many at first.. I'm guessing 20-30 at the most. data item.. one problem is what if two uses select the same item at the same time. Thanks for the help.

        A database would be your friend in this case.


        Just a tongue-tied, twisted, earth-bound misfit. -- Pink Floyd

Re: Data on the fly?
by tall_man (Parson) on Mar 01, 2003 at 04:22 UTC
    Really, this question is so vague it's not possible to answer. Let's take two sample cases and see which is closer to yours.

    The first is a web-page visitor counter. It's updated based on user selection, but there is very little data, with no structured organization. In that case a flat file would be fine.

    The second case might be an online catalog with many selections and prices. In that case, a database would make more sense.

      Thank you for your answer. The catalog is a closer match, difference is inventory would be low and if something is taken it needs to be reflected on the web site quickly that it's out. Where is a good place to look for info about setting up the database? Thanks. Qxotic.
Re: Data on the fly?
by pg (Canon) on Mar 01, 2003 at 05:20 UTC
    I have read the details you provided in your reply to data64.

    I would say, go database. There are lots of advantages you will get from database, but base on what you said, I want to stress one point:

    Although you just started, and it looks like the data structure is not complex, and amount of data is not big, you still should go database.

    You have to make your system ready for future, ready for growth. For sure, in the future, you will want to store more types of data, and the amount of your data will grow (maybe rapidly), database is definitely the technology provides you the flexibility and the power you want.

    For long term, database definitely requires less development effort.

      You have to make your system ready for future, ready for growth.

      Another option to make your system ready for the future is to make it as simple as possible. If it does exactly what it needs to do with no duplication, you'll have the flexibility to add necessary complexity.

        Another option to make your system ready for the future is to make it as simple as possible. If it does exactly what it needs to do with no duplication, you'll have the flexibility to add necessary complexity.

        Sure, provided that:

        1. You write unit tests to make sure that your necessarily complex additions (heh) don't break anything, and
        2. Your code is sufficiently decoupled that adding new components doesn't involve (excessive) changes to old.

        Both of these requirements are more easily met with simple software than with complex, "try to predict the future" designs. I need to tell myself that about once a week, on a good week. :-) I guess my point is that just making your software simple isn't enough, although it's a big help.

        --
        F o x t r o t U n i f o r m
        Found a typo in this node? /msg me
        The hell with paco, vote for Erudil!

        I absolutely agree with you that, making your system simple is one of the way to support growth.

        FoxtrotUniform gave a very good point: decoupling. I see decoupling as one way to modulize your application, and there are lots of other ways: put your code in a function, wrap your code in classes. You can even view multi-process, multi-threading as sort of modulization...

        Said all this, I am thinking whether we can combine your thought, FoxtrotUniform's thought and mine together, and say:

        To simplify the functionality of each module, and make modules have clearly and carefully defined/designed boundaries, would absolutely support future growth.
Re: Data on the fly?
by zemote (Scribe) on Mar 01, 2003 at 06:17 UTC
    Qxotic, now that you seem settled on a database you have to decide which database. Are you going open source or proprietary. Lots to choose from. I would recommend either MySQL or Postgresql if you are going the opensource route. Here is an article on MySQL and Postgresql compared that may help you decide (or not). http://www.phpbuilder.com/columns/tim20000705.php3 Guy's I know it's on another site that caters to that other "P" language, don't hold that against me. Hold it for the article's merits and not that other "P" language's flaws :) --zemote--
Re: Data on the fly?
by gmpassos (Priest) on Mar 01, 2003 at 18:46 UTC
    Use a database like MySQL. But if you don't want a DB server (only Perl), use SQLite, where the Perl module will work as the DB server. Take a look in DBI and DBD::SQLite.

    You can use DBD::CSV, but this is not a real databse, it only simulate it on files. SQLite is a real database, that is saved in a unique file.

    Graciliano M. P.
    "The creativity is the expression of the liberty".