IO::Socket::INET sets $@, not $!
I'm looking at IO::Socket::INET.pm, and it certainly does set $! under some circumstances, though it goes to great lengths to maintain $!, meaning that a previous value in $! can propogate through to snag the unwary.
Here's a simple way to duplicate the symptoms:
use IO::Socket::INET;
open(F, "<", "nosuchfile"); # set $!
my $socket = new IO::Socket::INET();
print "$!"; # prints "No such file or directory"
|