Your last line ("this creates a one way connection...") is incorrect-what you have done at that point is to make a connection to the remote server and authenticate. At that point, you can then do the following (adapted from the Net::SSH::Perl documentation):

my($stdout, $stderr, $exit) = $ssh->cmd($cmd);
# ... other code here ...
$ssh->cmd($cmd);
# ... other code here ...
[download]

Hope that helps.