But in any way, my users have to be able to connect to the database.
Do they really connect to the database, or just to a cgi application which then connects to the db? It sounds as though you're only using a 'master' user/pass to have db access from cgi. Do your users have individual db accounts? Can the CGI perform it's duties for userX if it is logged in as userX ? In that case you would be mindful of the security of your CGI session, (where user/pass is being flung around) instead of being worried about your db master password being available to an intruder.

Seriously, if someone invades the box, this is all academic - all bets on security are off. Please describe your process more explicitly, post some code and more people may be able to assist you.