Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^5: REST API with SFTP Pooling

by salva (Canon)
on Nov 16, 2020 at 16:05 UTC ( [id://11123698]=note: print w/replies, xml ) Need Help??


in reply to Re^4: REST API with SFTP Pooling
in thread REST API with SFTP Pooling

The way to pass extra options to the underlying ssh process from Net::OpenSSH is using the constructor master_opts argument.

The way to share the SSH connection is to pass around the path to the multiplexing socket (you can retrieve it calling method get_ctl_path).

You can also specify where you want to create the multiplexing socket explicitly (instead of letting the module pick a place for you) with the constructor argument ctl_path. Net::OpenSSH, by default, enforces some security policies for that path that you should obey.

So, for every connection you want to keep open (in your case, four connections every one using a different account), you need a process that establishes the connection and runs the dummy command from time to time. For instance:

use Net::OpenSSH; my $account_id = 1; my $ctl_path = "/.../some/known/place/to/put/the/control/path-$account +_id"; while (1) { eval { unlink $ctl_path; my $ssh = Net::OpenSSH->new($host, ..., ctl_path => $ctl_path); $ssh->die_on_error("Unable to connect to remote host"); while (1) { # run the dummy command $ssh->sftp->stat("/") or die "SFTP command failed"; sleep 10; } }; warn $@ if $@; warn "delaying before restarting SSH connection\n"; sleep 5; }
Then from the process where you want to do the SFTP operation:
... my $ssh = Net::OpenSSH->new(external_master => 1, ctl_path => $ctl_pat +h); my $sftp = $ssh->sftp // $ssh->die_on_error("Unable to create SFTP ses +sion"); # at this point $sftp is a regular Net::SFTP::Foreign object you can u +se in any way you like. ...

Most servers limit the number of channels open simultaneously in a single SSH session. For instance, it is 10 by default for OpenSSH. If you expect very many request coming in parallel, that could be a source of problems.

Finally note that allowing direct connections between a web server and other services has some security implications. If the front end becomes compromised, intruders would be able to access those backend services and the data there freely. Because of that, sometimes using an intermediate layer as the queue mechanism proposed in my OP could still be the most sensible thing to do.

Update: Both Net::OpenSSH and Net::SFTP::Foreign allow you to set timeouts for connecting and/or running commands. You should adjust those, to ensure that if any connection becomes stalled, it is discarded and established again, so that services recover promptly.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11123698]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (5)
As of 2024-04-25 14:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found