If you don't want to use a tunneling protocol for all the data, at least use an established, well tested system for exchanging the passwords.

Get yourself a Java PGP library to use in the clients, and a PGP perl module for the server. Require a private key in your servers conf file, and the corrisponding public key in your client conf files.