I would argue that physical security should be you're foremost consideration. I remember watching a rather amusing documentary a little while ago (British TV) following a kind of 'freelance' cracker. He described how one of the best sources of passwords and other sensitive information was the unmonitored garbage cans. In one scene he literally casually walked in(!!!) to an office building, laptop and equipment in hand, sat down in an office and began running packet sniffers etc.

After concluding his business, he walked out (at no point during his 30-or-so-minute stay was he confronted by a member of staff) and phoned the head office up to tell them what he had just been doing, and suggested they had a security problem.

Update: Oh, and if you want more example's of complete ineptitude when it comes to security matters, just take a look at the countless blunders attributed to the UK government...