As you have to create a Net::SFTP::Foreign object for every remote server, just use the correct arguments for every host.

$sftp1 = Net::SFTP::Foreign->new($host1, key_path => '/path/to/id_dsa'
+);
$sftp2 = Net::SFTP::Foreign->new($host1, key_path => '/path/to/id_rsa'
+);
...
[download]

You can also put that information on ~/.ssh/config (see ssh_config).