How can this system be defeated?

I can think of two ways: customers and management.

One of the big requirements of your system is that all your users have a PGP key. Most likely many of them won't, and many won't even know what one is. This will lose you business, and management tends not to like lost business. That said, you have to consider that security is a tradeoff. Will encrypting emailed passwords save you more money in security incidents than it will cost you in lost business? Things like consumer confidence in your company are hard to attach a dollar sign to, but most companies will try nonetheless.

The other security vulnerability is the customer themselves. If they do have a PGP key, but they pay no attention to security on their home computer, their account is still vulnerable. Modeling security threats with attack trees by Bruce Schneier is an excellent article that focuses on this concept. Also of limited vulnerability would be the users selected question. It could be easy to guess, or the attacker could convince the person on the phone they had made up a nonsense question and didn't know the answer.

All in all it's a good system. In a real-world environment you will have to make some security compromises in favor of usability, the hard part is deciding how far to take these compromises.