I think you need to show us more of your program - especially the part wher eyou create the ssh connections.

Also, the output of strace or truss would help.