I had to do something like this where I have to encrypt data on a public server, and very rarely have to decrypt a batch of encrypted entries in a one-off job.

After some time and thinking I found it easiest to use gpg. For encrypting I ran it with IPC::Run and the command-line that I used for encryption was:

gpg -a --no-secmem-warning --always-trust -r username -e
[download]

The always-trust is because I didn't want to worry about setting up the trust relationship so that it would accept the public key as valid - it is a public key, it isn't secret.

The private decryption I do by hand using gpg's decrypt-files to do a bunch at a time. Another co-worker is also able to decrypt the files, and the necessary key and instructions is also on a floppy in case the first two of us die.

This solution would not be acceptable if I needed to decrypt data more often.