I know this isn't answering your question about Term::ReadKey, but....

Have you considered using a GUI, a Graphical User Interface, like Tk, Wx, Tkx, Gtk2, etc? Just in case you weren't aware that better ways exit for Pong, than just using a terminal. See PONG with TkZinc for example.