This sounds like a great idea (but see below). As to your complication, I'd suggest taking a look at
Net::Telnet's get() command. I don't know exactly how you plan for your system to work, but if there's a space in it where you think there might be unexpected output from system X, you could get() for a certain timeout in those places. If the problem is more generally present while your program is running, it might be a good idea to have a loop, get()ing output from system X with a timeout and then reading input from the logging system.
I'm always thinking about this stuff, and you probably have too, but I feel compelled to mention that anybody who can telnet to your passthrough program will have access to machine X as the user you're logging in with. Telnet isn't the most secure protocol anyway, but it's worth thinking about.
Overall, this sounds like a pretty cool solution to me.
--
Love justice; desire mercy.