Generally, $ENV{DISPLAY} isn't going to give an exact port number. The typical X11 setup assumes (TCP 6000 + $port) where $port is what you find in the variable. There are more complicated port configurations like "localhost:1.1", though, so I didn't go into lengthy discussion. Use the X11 modules which presumably do it right.
Theat's the part that's untested. Damn, I remembered the colon, but forgot that wat's after it is frequently some number that would be silly at best as a port number. (I think I confused it with the way X connections tunneld in ssh look in netstat output, which is on the order of localhost:6012). DRAT!
Given a correct IP and port, the rest of the code should be good because it's extracted from code I actually use. But the ($addr, $port) = split(/:/, $ENV{DISPLAY}) is almost certain not to yiled a good port.