You've gotten several good answers on the problem you think you have. But you have a larger problem.

If this Perl code is really meant for linguistic research, (as opposed to being a toy program for a computational-linguistics 101) don't roll your own parser / tokenizer. Already your approach embodies design-decisions that will bite you.

• It assigns each token to a single linguistic category; ignoring polysemia. What will it do with 'back', which can be a noun, verb, adverb, adjective or preposition?
• Sometimes you'll want to tokenize across whitespace. Eg, 'break up' is better classed as a verb than as a verb+preposition token.

I encourage you to check out IBM's (free, open-standard) UIMA.

throop