You might get more information if you print out the value of $! if the connection fails, but this sounds like an OS 'feature', you might want to check here to see if there is anything that fixes it.

/J\