On two? Does it do this on other's as well or only those two? Might help resolving what's going on here
I did see something on the perldoc page suggesting it can read identity files. I haven't done it this way myself however. Have you tried removing the RhostAuthentication option in your code? Everything else looks similar to what I've seen done. Just a thought.