Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Sysadmin Aspects of DBMS

by jlongino (Parson)
on Nov 23, 2001 at 23:44 UTC ( [id://127166]=perlmeditation: print w/replies, xml ) Need Help??

I'm currently looking into using Perl for DBMS related projects and would greatly appreciate input from other monks on the sysadmin aspects of installing and maintaining the various DBMS.

After a few days of SuperSearching, I gathered that PostgreSQL is the most fully functional/complete of the various options (while not necessarily the fastest or best suited for all tasks). I went ahead and downloaded/installed PostgreSQL (since the docs said it was required) and the DBD::Pg driver module.

After it was installed, I tried to start the server and found that there were several directory permissions snags that I had to sort through just to get past the "permission denied" messages.

Once I got past the permissions problems, I got the message about needing to increase SHMMAX value. After researching this via SunSolve (this is on a Solaris system) I found the correct syntax for changing the /etc/system file and did an "init 6" to restart the system (5:30 PM Thanksgiving day and I was the only user logged on). Well, 30 minutes later the system wasn't up yet so I had to make a trip to the machine room. Seems the root partition needed an fsck. Which would've happened regardless of the /etc/system changes but a hassle nonetheless. Three hours later it was back up and running again.

So the question is do the other DBMS require this much hassle (requiring server installations, system memory tweaks, etc.) or was this experience typical? Seeing as I've already spent three days researching PostgreSQL (which seems to be working for the moment) I would appreciate some feedback from other sysadmins that have been through several different DBMS installs and can enlighten me as to what types of regular maintenance I should expect (the costs usually hidden from the end user/programmer).

TIA for your time.

--Jim

Replies are listed 'Best First'.
Re: Sysadmin Aspects of DBMS
by thraxil (Prior) on Nov 24, 2001 at 01:49 UTC

    i've worked with PostgreSQL and MySQL extensively and lately i've been doing a lot with DB2.

    i've never really had any problems with either postgres or mysql on linux. both installed easily and worked fine right out of the box. DB2 (on linux and solaris) has been a royal pain in the ass in every way imaginable.

    as far as maintenance, pretty much all you should need to do for postgres is a nightly vacuumdb (for performance). also keep in mind that DBMS's often have special requirements for backups. sometimes just backing up the binary data files isn't good enough. to be safe, i run a pg_dumpall every night before the backup scripts run. this writes a copy of all the data out to a text file that you can use to restore things pretty easily. of course, i work with a bunch of really small databases; this might not be such a good idea on really big databases.

    basically: postgres and mysql are both pretty easy to admin as far as databases go as long as your needs are fairly simple. if you're working with enormous databases under heavy load with all kinds of unusual requirements, i'm sure you'll spend a lot more time admining the DB than you'd like but that's going to be true no matter what DBMS you use.

    (plus, i believe O'Reilly is coming out with a postgres book very soon. joy.)

    anders pearson

      Thanks Thraxil,
      This is just the type of info I was hoping for. I did some dbadmin stuff way back in '86 (?) for oracle on an IBM 4341 and it had similar maintenance requirements.

      My applications won't be large scale, but they will be moderately complex (employee demographic/deparmental info) requiring multiple related tables. The user load will be small (<=6 users). I will be exporting/dumping the composite data regularly to text files anyway since several applications hang off this data and it will take some time to convert them all to use the database. Most of the tables will be refreshed weekly from text files anyway.

      I'll be looking forward to the O'Reilly book.

      --Jim

        "[R]equiring server installations..." I can only imagine you're used to working with flatfiles from that line ;). I'm mainly going to mirror Thraxil here, I've worked with Oracle, DB2, and PostgreSQL extensively and PostgreSQL does have a rather light maintenance load.

        How much work needs to go in will of course depend on your load, dataset, and the equipment you're running on. For many the defaults will be sufficient. Personally, even for light loading, I like spending a bit of extra time on install and tweaking everything to the hardware. Obviously you can do quite a bit partitioning, managing tablespace, etc. but

        From the sound of your data/load setting up a couple cron jobs and applying the occasional patch will probably be all the sysadmin work you'll need to invest in it. VACUUM and VACUUM ANALYSE (or the command line wrapper vacuumdb as noted by Thraxil) and some backup cron jobs will probably be the extent of your involvement.

        If you're going to be the DBA as well then an occasional audit of expensive queries (using EXPLAIN) and indices maintenance will often prove fruitful; though not required unless you're experiencing poor performance. Again, this is based on what you've said of your application.

        To get a peak at the O'Reilly book before it's published (or to preorder), Command Prompt (the writers) have it online here. Bruce Momjian's PostgreSQL: Introduction and Concepts is also available online. And you can of course always read the official Administrator's Guide.

        P.S. You may want to ask the editors to put an [OT] in the title of the post.
Re: Sysadmin Aspects of DBMS
by mpeppler (Vicar) on Nov 24, 2001 at 04:40 UTC
    I've done quite a lot of Sybase admin, and tweaking the OS to get the better performance for any RDBMS is normal.

    As others have mentioned you should at first concentrate on backup/restore procedures, and then on the general system maintenance (such as space usage, index maintenance, etc).

    I don't know PostgreSQL, but for Sybase there are a few "standard" operations that you'd need to do, such as full db dumps, transaction dumps (i.e. incremental dumps), index statistics maintenance, database consistency checks, and table reorg operations.

    All of these can be easily automated via simple perl (or shell) scripts.

    Michael

Re: Sysadmin Aspects of DBMS
by dws (Chancellor) on Nov 24, 2001 at 02:42 UTC
    Beyond the relative simplicity of installing and configuring an RDBMS, it's worth looking ahead at what the backup and recovery procedures are, since those costs will eventually draw install. With many systems, you can't rely on a standard backup utilities, and have to use RDBMS-specific ones. (Since a DMBS can change multiple files as a side-effect of a single query, making sure those files are in a consistent state when archived is very important.)

    I administered a SQL Server 6.5 system for a while. The built-in archiving tools were adequate, and our database fit on a single DAT tape, allowing archiving to run fully automated. All I had to do was remember to swap tapes.

Re: Sysadmin Aspects of DBMS
by edebill (Scribe) on Nov 24, 2001 at 20:35 UTC

    I think you've just discovered why there are people who's job title is "Database Administrator". MySQL and Postgresql are relatively easy to administer, but all the big commercial databases can get fairly involved, as you tune performance and try to get smoother backups (you can get fairly deep into the free ones, too).

    I'm a developer and sysadmin who's gotten dragooned into being the DBA, and I have to say that DB2 is just as complex as the OS it runs on - only the defaults aren't as good. Having watched the DBA's at other places, and talked to people using other RDBMS's I'm pretty convinced that all the commercial ones are fairly rough in their own way.

    As for having to tweak the OS, well, it seems to me that Un*xes tend to ship configured for people running lots of small programs, so you end up doing a bit of reconfiguring in order to run something big like an RDBMS. And the more you try to do with it, the more tweaking you need.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://127166]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (4)
As of 2024-03-29 09:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found