First of all, you are not supplying the password for the user, which will give you the local: Trying empty user-authentication request. error message. Secondly, IMHO, ssh_args is a list of named arguments, and therefore a hash. You might want to change

my %args = (ssh_args => [] );
...
push @{ $args{ssh_args} }, port => 22;
[download]

to

my %args = (ssh_args => {} );
...
$args{ssh_args}->{port} = 22;
[download]

The revised code shold look somewhat like this:

use Net::SFTP;
my %args = (ssh_args => {});
$args{user} = $user;
$args{password} = $pass;
$args{ssh_args}->{port} = 22;
my $sftp = Net::SFTP->new($host,%args);
[download]

Everything above is untested, but I think that it'll work for you. Let me know if it does not.