packet_write_wait: Connection to xx.xx.xx.xx port 22: Broken pipe

TusharK has asked for the wisdom of the Perl Monks concerning the following question:

Hello, My application runs in multiprocessing and I am using "Net::OpenSSH" module to connect to the Cisco routers via SSH.This module is running for a single router successfully. But when I am executing the process (with the 16 child processes) for around 20 thousand devices, the processes are getting killed randomly. When I reviewed the application log file I found the following exception: "packet_write_wait: Connection to xx.xx.xx.xx port 22: Broken pipe". Could someone please suggest what I am doing wrong here and how to resolve this issue?

Comment on packet_write_wait: Connection to xx.xx.xx.xx port 22: Broken pipe

Replies are listed 'Best First'.
Re: packet_write_wait: Connection to xx.xx.xx.xx port 22: Broken pipe by haukex (Archbishop) on Sep 14, 2020 at 11:42 UTC
That error means that the ssh connection was broken, and it doesn't really have anything to do with Perl. It could be for lots of different reasons, but often with SSH, the connection is dropped due to inactivity - if you're doing 20k devices from 16 processes, it's possible one of them is getting starved. Some commonly given advice for this issue is to put the following in your `~/.ssh/config`: `Host * ServerAliveInterval 60` [download] However, with this many connections to this many devices, it's also possible the issue is somewhere else, and that it either has to do with the large amount of connections you're opening, or that those devices you're having trouble with actually have issues on their end. You can try running a Wireshark trace and then filtering by IP afterwards with the IPs that were giving you trouble, so you can possibly get an idea that way.	[reply] [d/l] [select]
Re: packet_write_wait: Connection to xx.xx.xx.xx port 22: Broken pipe by salva (Canon) on Sep 14, 2020 at 13:12 UTC
`packet_write_wait: Connection to xx.xx.xx.xx port 22: Broken pipe` That means the remote host has closed the connection. That is usually due to a timeout or some protocol interoperability issue. You are not showing us any code, so it is difficult to give you more specific advice.	[reply] [d/l]
Re: packet_write_wait: Connection to xx.xx.xx.xx port 22: Broken pipe by perlfan (Parson) on Sep 14, 2020 at 16:16 UTC
>(with the 16 child processes) for around 20 thousand devices Congratulations, you've implemented a network & services monitor. Best bet here is to be able to detect and recover or retry whatever it is you were doing when the connection was broken or failed.	[reply]