Are you sure you know how PGP works? The users have to send you their public key and you use that for encryption. Then they can decrypt with their private keys.

That's how I understand it. Feel free to prove me wrong ;-)