jpsartre has asked for the wisdom of the Perl Monks concerning the following question:

I have to build a file serving system for hundreds of clients per implementation.
The data served is sensitive in nature- Bank type stuff.
I have a lot of questions but I'm having a hard time knowing where to turn for peers and good advice.

Some examples of questions i have are:

-What I need advice on..
Where do I go to ask these questions ? I am asking here because the backbone of my existence is perl. (im just a newbie, and allways will be). *nix people are solid. So I am asking here for advice on .. where to get advice.

Replies are listed 'Best First'.
Re: Where to get this kindof advice.
by Corion (Patriarch) on Oct 25, 2005 at 13:20 UTC

    Whatever legal departement you have at hand, get them involved. Financial transactions involving more than one party will inevitably lead to lawsuits, and you want the legal departement to take the blame instead of taking the blame yourself. Things to consider:

    • Where is the data sink? At which point does your client consider the transaction to be in your hands? You want to make sure that you give out the positive acknowledgement only after you've passed on the data to the next system for processing. This is not always possible, and then your system runs the risk of confirming transactions it has no chance to pass onwards.
    • How easy is it for one client (or bad employee, in conjunction with a well-paying client) to fake data for the/another client? HTTPS (or SSL) should be the minimum here to prevent sniffing. If it's not possible to convince the customer otherwise, see that you get the appropriate disclaimers signed in paper and that you document when you advised them, preferrably in paper too.
    • Really think about where your system is the authority on things and where not. Consider using a directory (LDAP) instead of managing the data yourself. If you manage the data yourself, you open up your system to synch-lag - if an user is disabled in the main directory she might still have/gain access to your application, which I consider bad.
Re: Where to get this kindof advice.
by pboin (Deacon) on Oct 25, 2005 at 14:03 UTC

    My authority suggests forcing ssl on these implementations should be optional, and we should let a client sign a waiver to use the system without ssl. This is a hole like Satan's *blank*hole. Am I being a reputable professional if I give in to these demands?

    You are a reputable professional if you educate your employer (adjust definition appropriately) of the risks. *Always* tell people how it is, even if it causes moaning, bitching and ego bruises. Be tactful, but get the truth out there -- that's what makes a professional.

    Now, the ball's in their court. He who pays the bills calls the shots. If you feel strongly enough, it's time to work for someone else.

    I want to make the filesystem be the authority on a lot of things, like tracking users - for example, if a flat file called "joe" exists in a project directory, then user joe can enter it- (other people will be maintaining the system later, i need to ease it for them) So.. this info will be fed into a db for quick querying.

    So therefore.. what should be my main authority on users, the db or the filesystem?

    Personally, I like all my eggs in one basket -- stick it in the DB. Besides relational databases have a lot more functionality than filesystem metadata. You might have to front-end it for your admins, but if this is a banking application, that should be expected. Also, having all info in the DB makes distater recovery / backup a bit simpler I would think. Your DBA does his job, backs up the dump and you're good.

    Good luck. Sounds like you're in a unique environment...

      I want to second that about storing the authentication data in the DB.

      (-: If you have to, alias the "ls" command for the admins.

      Write a command that looks into the db, writes out what users have access -- and then exec's "ls" with the in-parameters... :-)

Re: Where to get this kindof advice.
by sauoq (Abbot) on Oct 25, 2005 at 13:54 UTC
    Am I being a reputable professional if I give in to these demands?

    Who is making the demand? Are you an employee? Contractor? It sounds like the requirements are at fault, not you. You can disagree with the requirements all you like, but if they are requirements and it is your job to implement them... well, that should answer your question. You can question why, suggest alternatives, and shake your head in disbelief all you want, but at the end of the day, a professional is going to give his client what the client needs. Just make sure it is all written out very clearly (and record your misgivings) so that, if it all goes horribly wrong, at least you will be absolved.

    So therefore.. what should be my main authority on users, the db or the filesystem?

    Keep the auth data centralized. Your idea of using a file called "joe" to provide access to the project that file resides in is an unsecure and unmaintainable mess waiting to happen.

    -sauoq
    "My two cents aren't worth a dime.";
    
Re: Where to get this kindof advice.
by Perl Mouse (Chaplain) on Oct 25, 2005 at 14:39 UTC
    Am I being a reputable professional if I give in to these demands?
    I don't think that's a sensible question. What's your role in the entire project? There's a big difference whether you're just a junior code monkey in a team of 50, with tech leads and architects, or whether you basically carry the entire project, and it's your ass that's on fire when a third party runs off with the clients, or worse, the banks money. And what you haven't discussed at all is the sensibility of the data you are serving. Since you are talking about files, this smells like static data. Which may have data of only limited sensibility. There's a big difference between an application that lets a customer move a million dollars from one account to another, and a application that shows teenagers the amount of money in their checking account.

    So therefore.. what should be my main authority on users, the db or the filesystem?
    Probably the database, but to be sure, one needs to know what filesystem and what database. Most databases care a lot more about data than most filesystems, but some databases trade reliability for speed, and some filesystems trade speed for reliability.
    Perl --((8:>*

      Wow. All this feedback is tremendously useful. I really can't thank enough.

      Thank you for the arguments on centralizing data, and the questions I need to ask myself/etc on this project.

      I am in charge of the project, I am an employee, and the project is made first for my company (and implemented for our use)- and then to be offered to other companies of similar needs with their clients.. (Sound familiar?)

      I have full control with security in the first implementation of this project/system. But from there on out.. It's going to be a little bit out of my hands, and I am near vomiting that levels of security (on/off) will be at the mercy of salesmen. It makes me seriously ill thinking about it.

      Yes, my ass *is* on fire if things go wrong.
      It's hard because the company is very white collar clean cut .. just plain white here and there. So .. I have to be very dipplomatic about what we should and should not be doing, and what we can and can not be doing.

      Honestly I want this to work well and reliably for all involved, I'm not simply making sure to stay out of the firing range.

      It makes sense to make the authority be the DB. I was seriously considering using the structure of the filesystem to populate the database, and then later on the database would be always just a quick way to see filesystem data.
      That is, if empty flat file "joe" were in directory "my stuff" it would make the right entry in the projects/users table.
      I was scared about the prospect of the dabatase being corrupted. I still kind of am. It seems like an awful lot of stuff on one file (the db(s)).
      The info is sensitive. Most of it is financial/classified data for insitutions.

      I will build/offer the full security platoon for my company, and then make sure paperwork needs to be signed before the little soldiers are put to sleep one by one- for other companies.

        It's hard because the company is very white collar clean cut .. just plain white here and there. So .. I have to be very dipplomatic about what we should and should not be doing, and what we can and can not be doing.

        No, you don't. You can and should speak plainly. You just have to communicate in terms that businessmen can understand. Speak in terms of risk, liability, and accountability. Those are things that are important in business. Computers, networks, and security typical aren't; unless they impact the bottom line on the company balance sheet.

        Specifics to address should include:

        1) You're not willing to be held accountable for failure of this system unless you have authority over how it will be implemented. Pointed refuse to take the blame for someone else's business decisions.

        2) You're concerned about the risks and the associated costs the company is exposing itself to by it's poor security. Ask for a written explanation as to why they're not adopting a more secure risks-management policy, and point out, in writing, exactly how much money they stand to lose if something goes wrong. If it's a trivial amount, then perhaps they don't care. If it's not (and it's probably not), point it out to them. Remember, businessmen don't care about computers; but they care a lot about money.

        3) If you're worried about costs of errors (security settings being set wrong), point that out, as well. Develop best case, worst case, and realistic probable case scenarios, with numbers. Prove your points to the senior management.

        4) If they still won't listen to you, object formally. Get requests you disagree with put in writing, then implement them as written. Get clarifications in writing if anything is unclear. Once they formally order you to do something, it's not your decision anymore, and neither is the blame for that decision. That "pulls your ass back out of the fire", so to speak. By getting your orders in writing, you're making the point that you disagree strongly, but will comply so long as your boss takes the blame for his own decisions. If he won't do that, one or both of you should leave the company. ;-)

        Don't be afraid to speak up if you think there's a significant problem. On the other hand, be prepared to back up your fears with a proper risk analysis: what you think can go wrong, why it might go wrong, and how those risks would be mitigated if they did things "your way". Learn to make the business case for what you want; it's a skill that will serve you well throughout the rest of your life, even if management rejects it for this project.

        I suggest you think about auditing as well - if management has any competence at all in the business processes, they should find the need for an audit trail readily apparent. You need to have the programme write an audit log as it goes along.

        You could use this as an 'in' to get security onto the agenda - e.g. what's the possibility that a change can be made without any record of who made it? and (even if there's no fraud) what would the company's auditors think of that?

        HTH!

Re: Where to get this kindof advice.
by gloryhack (Deacon) on Oct 25, 2005 at 18:43 UTC
    Your employment contract, generally otherwise worthless in most cases, should contain a hold harmless clause -- make sure it does. If you have any doubt, you might consider paying an attorney to review your employment contract and advise you about the applicable statutes. If you consult with an attorney, keep it secret (even from your cow-orkers) so you don't prompt an overreaction from your employer.

    IANAL, but: Assuming you're adequately protected by a hold harmless clause, you should be covered unless you do something you know or should know is illegal. Ethics might be personally bothersome, but any legal liability for unethical conduct falls upon your employer as long as you personally don't break the law.

    If you were a third party (consultant|contractor) I'd suggest walking away from the project if the security specifications don't fall within your risk acceptance comfort zone. If you were a third party, you would have to meet a much higher standard in order to maintain the legal shield, because your client would be considered to be "acting in reliance upon your professional judgment". It's generally not a defense to show that you had to choose between doing a stupid thing and losing the contract.

      Assuming you're adequately protected by a hold harmless clause, you should be covered unless you do something you know or should know is illegal. Ethics might be personally bothersome, but any legal liability for unethical conduct falls upon your employer as long as you personally don't break the law.

      Also IANAL but I do work in the banking sector. Much depends on the laws of the country in which the OP is operating. But, banking is a global market, and you should know something of other regulatory bodies outside your own country, as they will have an effect.

      The USA has the Sarbanes Oxley Act of 2002, which makes hiding and misrepresentation of financial information, a felony, as indeed is having knowledge of such and doing nothing about it. This affects third parties including contractors and auditors.

      SOX has provision for protecting whistle blowers, but this is probably a part of the legistlation which has not been invoked, owing to the newness of the act. Also, be aware of some of the dangers of whistle blowing, as in the Chip Salzenberg case.

      Outside the US, is the Basle Accord, in particular Basle II, which covers practices for banks which need to operate in Europe.

      Some pointers for you:

      • Research Sarbanes Oxley and work out what implications it has for you
      • Research Basle II (which is also written Basel II, depending whether the paper is of Franco-Swiss or Germanic-Swiss origin)
      • Follow the Refco case, read the news feeds as it happens, study the legal aspects
      • --

        Oh Lord, won’t you burn me a Knoppix CD ?
        My friends all rate Windows, I must disagree.
        Your powers of persuasion will set them all free,
        So oh Lord, won’t you burn me a Knoppix CD ?
        (Missquoting Janis Joplin)

Re: Where to get this kindof advice.
by nothingmuch (Priest) on Oct 26, 2005 at 08:54 UTC
    I think the wisest choice is to make a rich authorization system.

    The most adequate design I can think up is one where each operation triggers a hook that makes a query into the system to find the needed attributes for that op.

    The other part is the object representing the user's session - it also has attributes. Since you speak of this virtual filesystem that owns all the data in the project attributes on users and attributes on parts of the filesystem can be embedded in the filesystem too. This may sound tempting, but I'd be weary of overabstracting.

    Then you should design it to make it easy to do two things:

    • Make the system that checks what attributes are needed for an op check the user object too. Then make the "private channel" attribute optional, on by default, and dependant on that client waiver.
    • Make SSL connections (and similar transports) enable the "private channel" attribute on the context object
    This way you can change add this functionality later on when it has been decided on. If they change their mind later, this feature can be easily removed. If something new comes up, it'll be easy to adapt to it.

    It's a sad fact, but all layers of abstraction can sometimes have an unforeseen limiting factor. If your legal department can handle the waiver safely i think making SSL optional a good idea, since it will give your clients more choices on how they want to work with you. I doubt this will come up in practice that often.

    Lastly, it might be wise to use a certificate system to ensure that the authz process verifies that the waiver is cryptographically signed by the client (where a the cryptographic signature can be made on behalf of the client when they physically signed the waiver), and that the signature is signed by an authority (or organizations operators). That way at least the protocol is very clear and well defined.

    -nuffin
    zz zZ Z Z #!perl
Re: Where to get this kindof advice.
by Moron (Curate) on Oct 26, 2005 at 09:35 UTC
    The customer is always right; and at the level where waivers are getting signed, us mere technical mortals would be overstepping our bounds (hugely!) if we interfered at the sharp end of the business.

    The best anyone can do is to make sure the customer takes responsibility for what they exactly order, which in this case someone has obviously thought of already.

    Even if it were a situation where your opinion was the best one for the business, it doesn't mean it's necessarily your call or even the head of your department. It seems more likely something the legal dept. or compliance would be responsible for - perhaps it was their advice to get the waiver in the first place. And then there's the fact that mostly in practice the lowliest salesman has the main authority when it comes to someone from another dept. coming along and trying to stomp all over their deals (more so, for done deals).

    Update: beware also of colleagues who try to goad you into precipitous action that will get you fired (even held responsible for defaming the organisation) and instead abide by the decision of your own manager, 'just following orders' being the best panacea for this not being your problem.

    -M

    Free your mind

Re: Where to get this kindof advice.
by zentara (Cardinal) on Oct 26, 2005 at 11:57 UTC
    Bank stuff....forcing ssl on these implementations should be optional, and we should let a client sign a waiver to use the system without ssl.

    Yes, my ass *is* on fire if things go wrong.

    Wow, it sounds like your bosses are setting up a scam and positioning you to take the fall when the FDIC comes looking for the money. I would demand that a legal document be drawn up absolving you of any responsibility. Hire a lawyer.


    I'm not really a human, but I play one on earth. flash japh
Re: Where to get this kindof advice.
by pajout (Curate) on Oct 26, 2005 at 12:40 UTC
    Imho, enable non-ssl connection or another 'features', but exactly (and unarguably?) inform the customer representative (and/or your authority) about security impact.
    And what about main authority... Remember Decalogue... You will have only one authority. I think that filesystem will be better choice, and, in this case, think about db as about cache. Or, if you will choose db as authority, think about filesystem as about backup-layer or mirror. The point is where updates could be done.
Re: Where to get this kindof advice.
by tirwhan (Abbot) on Oct 27, 2005 at 09:32 UTC

    I'm curious, does the person suggesting the non-SSL option have any reason for why people should want this? I can't think of one. The idea of transferring any kind of financial data over an unencrypted connection give me the shivers to be honest.

    Regarding the authentication authority, if you've got a mixed environment where you need both a filesystem and a database to work off the same authentication data you should take a look at PAM, which allows you to use a backend of your choice. Most databases can authenticate against PAM, and there are various CPAN modules for this. Using a tried-and-tested authentication layer will surely make your app more secure. The PAM mailing list may also be a good place for specific questions you have.