Are these unencrypted private keys, are you being prompted for the passphrase or is this the problem:

When loading each of the private key files, the client first tries to load the key using an empty passphrase. If this fails, the client either prompts the user for a passphrase (if the session is interactive) or skips the key altogether.

--Net::SSH::Perl::Auth::PublicKey