If you have the Gtk2 system installed, you might want to look at it's base object class called Glib. Read "perldoc Glib". Glib itself does not need the Xserver, and it has alot of powerful minds actively working on it.

ABSTRACT
This module provides perl access to GLib and GLib’s Object 
libraries.  GLib is a portability and utility library; GObject
provides a generic type system with inheritance and a 
powerful signal system.  Together these libraries are used as
the foundation for many of the libraries that make up the
Gnome environment, and are used in many unrelated
projects.
[download]

I have a simple example at Roll your own Event-loop