Myself, i have no perl/Gtk2 installed, but i agree with your approach. Who really needs numeric keypad functionality or something on the console when Tk or Gtk will do this nicer and easier in X?
Still, there can always be something learned from trying. But if the gods have ears may this save someone else a silly amount of time & everyone can move on.
Or maybe a better Term::Readkey might come to exist.