Re^4: Net::OpenSSH connection failing with 'muxserver

Yea, I missed the forked call to the bind. It's here:

113710:    _poll(0x2FF1C1D0, 1, 2000)            = 1
113710:    nrecvfrom(4, 0x2FF1E0F0, 1024, 0, 0x2FF1C970, 0x2FF1C1C0) =
+ 133
113710:    close(4)                    = 0
113710:    socket(2, 2, 0)                    = 4
113710:    getsockopt(4, 65535, 4104, 0x2FF1DE14, 0x2FF1DE10) = 0
113710:    connext(4, 0xF027F2E0, 16)            = 0
113710:    send(4, 0x2FF1EC00, 41, 0)            = 41
113710:    _poll(0x2FF1DE90, 1, 2000)            = 1
113710:    nrecvfrom(4, 0x2FF1F4C4, 1024, 0, 0x2FF1E630, 0x2FF1DE80) =
+ 154
113710:    close(4)                    = 0
113710:    access("/etc/krb5.conf", 04)            = 0
113710:    statx("/opt/krb5/etc/krb5.conf", 0x2FF1FB58, 76, 0) Err#2  
+ENOENT
113710:    open(0xD0ADB328, O_RDONLY)            = 4
113710:    fstatx(4, 0x2FF20CF0, 76, 0)            = 0
113710:    kread(4, " Ù _9F <80 T } ¿ Ú ± Q ½".., 20)    = 20
113710:    close(4)                    = 0
113710:    _getpid()                    = 113710
113710:    open("/opt/dcelocal/var/security/creds/dcecred_6e245c24", O
+_RDONLY) = 4
113710:    kfcntl(4, F_SETLKW, 0x2FF208A0)            = 0
113710:    kread(4, "0501", 2)                = 2
113710:    lseek(4, 2, 0)                    = 2
113710:    kread(4, "\0\0\002\0\0\00F r c h l".., 1024)    = 1024
113710:    lseek(4, 2, 0)                    = 2
113710:    kread(4, "\0\0\002\0\0\00F r c h l".., 1024)    = 1024
113710:    lseek(4, -989, 1)                = 37
113710:    lseek(4, 37, 0)                    = 37
113710:    kread(4, "\0\0\002\0\0\00F r c h l".., 1024)    = 1024
113710:    lseek(4, -574, 1)                = 487
113710:    kfcntl(4, F_SETLKW, 0x2FF20C80)            = 0
113710:    close(4)                    = 0
113710:    access("/etc/krb5.conf", 04)            = 0
113710:    statx("/opt/krb5/etc/krb5.conf", 0x2FF1FC68, 76, 0) Err#2  
+ENOENT
113710:    open(0xD0ADB328, O_RDONLY)            = 4
113710:    fstatx(4, 0x2FF20E00, 76, 0)            = 0
113710:    kread(4, "94 ? 7 È k i È : à § ¼ Ç".., 20)    = 20
113710:    close(4)                    = 0
113710:    _getpid()                    = 113710
113710:    access("/etc/krb5.conf", 04)            = 0
113710:    statx("/opt/krb5/etc/krb5.conf", 0x2FF1FD28, 76, 0) Err#2  
+ENOENT
113710:    open(0xD0ADB328, O_RDONLY)            = 4
113710:    fstatx(4, 0x2FF20EC0, 76, 0)            = 0
113710:    kread(4, " Å8A e a ±9F ¤8319 u È !".., 20)    = 20
113710:    close(4)                    = 0
113710:    _getpid()                    = 113710
113710:    kwrite(3, "1D Ø8C Ï a ( > · k [12 <".., 80)    = 80
113710:    _select(4, 0x20043210, 0x00000000, 0x00000000, 0x00000000) 
+= 1
113710:    kread(3, "7F82 à ? $ v s Ý ® · l %".., 8192)    = 32
113710:    socket(1, 1, 0)                    = 4
113710:    umask(127)                    = 18
113710:    bind(4, 0x2FF21030, 87)                Err#1  EPERM
ssh_control_listener bind(): Not owner
113710:    kwrite(2, " s s h _ c o n t r o l _".., 40)    = 40
113710:    _exit(255)
[download]

Nothing stands out to me as a possible previous error that might be causing the bind error. I did see the following repeating error in another process that may also be related:

477272: kioctl(3, 22528, 0x0000000000000000, 0x0000000000000000)113710
+: ENOTTY
[download]

Seems like that'd be expected though since ssh is being run through a Perl module.

Permissions on .libnet-openssh-perl are set to 700.

Comment on Re^4: Net::OpenSSH connection failing with 'muxserver_listen bind(): Not owner' Select or Download Code

Replies are listed 'Best First'.
Re^5: Net::OpenSSH connection failing with 'muxserver_listen bind(): Not owner' by salva (Canon) on Sep 01, 2011 at 22:14 UTC
AIX truss may accept some option to make it also dump syscall arguments (as those passed to bind). Otherwise a debugger could be used to inspect them, but this is quite more difficult. Calling SSH with the verbose flag may also give some useful information: `$ssh=Net::OpenSSH->new($host, master_opts => '-vvv',...)` [download] Finally, witch version of OpenSSH are you using?	[reply] [d/l]
Re^6: Net::OpenSSH connection failing with 'muxserver_listen bind(): Not owner' by Only1KW (Sexton) on Sep 02, 2011 at 13:56 UTC
I don't see any option for truss that would dump the arguments. Verbose doesn't offer any new help: `debug1: Authentication succeeded (keyboard-interactive). debug1: setting up multiplex master socket muxserver_listen bind(): Not owner` [download] `~> ssh -V OpenSSH_5.1p1, OpenSSL 0.9.8j 07 Jan 2009` [download]	[reply] [d/l] [select]
Re^7: Net::OpenSSH connection failing with 'muxserver_listen bind(): Not owner' by salva (Canon) on Sep 02, 2011 at 15:35 UTC
The only thing that comes to my head is that the multiplexing socket path could surpass the 108 chars of the sockaddr_un struture passed to bind. You could try running ssh from the command line enabling the multiplexing feature with different paths, for instance: `ssh -x2M -S /tmp/mux localhost ls` [download] If that works, you could try with the exact same command executed by Net::OpenSSH. You will be able to see after enabling debug mode on your script: `$Net::OpenSSH::debug=-1;` [download] Besides that, the other thing that can be done is running ssh under the debugger in order to see what is being passed to bind. I would do it myself, but have been unable to find any place offering free AIX accounts on the internet.	[reply] [d/l] [select]