Net::SSH::Expect is not completely reliable because of the way it sends commands to the remote machine talking to a shell. Try using Net::OpenSSH instead.

In any case, a likely cause for your problem is that the restart.sh script may leave behind processes with some of their stdio streams attached to the SSH channel. In that case the ssh process will wait for those processes to close their streams before returning control.