Firstly you possibly could just use a normal hash, since the same socket should (maybe?) always stringify to the same thing (since the object will be in same memory location until destroyed), thus allowing you to do it the simple obvious way.

Otherwise you could create a subclass of IO::Socket which allows extra data to be stored in the object, or just use the syntax on a normal IO::Socket:

my $sock = IO::Socket->new(...);
${*$sock}{my_connection_data} = 3;
...
print ${*$sock}{my_connection_data};
[download]

I found that syntax in the IO::Socket code a while ago when I was trying to do much the same thing, however I can't remember how well it worked when using IO::Select and everything. Of course it may not be the `right' to play with the reference like this. And this method will likely be broken with Perl 6.