Yeah, it's enough to make you scream, isn't it? :-) At least the key for encrypted swap is somewhat protected (you pretty much have to be root to get at it, assuming there aren't gaping bugs in the swap or encryption layer). There's a point at which paranoia has just got to stop, and one needs to say "secure enough", I guess.