Look at Term::ReadKey. I think that might be what you want, or if you're on MSWin Win32::Console may be relevant. Term::ReadKey has an example very much like the one you mention.

Both modules are mentioned in the recent thread Re: Getting special keys to work with Term::ReadKey in Windows