Generally speaking, your idea would fly. (It is not clear to me whether your main concern is that you used different languages at different nodes. that should be okay, TCP itself has nothing to do with programming language)

I once did something called message router, which is a set of libraries for a private user level protocol used in my shop. The library has three versions, one in Perl, one in c, and one in java, so all the other programmers in my shop can call it, doesn't matter whether he/she is a Perl/c/Java programmer.

Those libraries worked fine.

In your case, I suspect that it might be better if you create one more level of abstraction on top of TCP, to handle your user level message routing. Not sure, depends on your requirement, what I said is just my experience and my solution at that time.

As for IO::Socket::INET, it is just fine, that was what I used.

If you ask me to revisit it again today, considering the tech progress people made between then and today, I might like to study Java Messaging a little bit, if Perl is not a must. That might help avoiding reinvent the wheel, and gain a more sophisticated solution.