Since you are dealing with numbers you could just set the value of an array element to its' index. Instead of getting rid of a selected value, you could set it equal to undef when selected.

A benefit to doing this way would be the minute amount of code needed to maintain your array.

This is probably not the best way to handle this programatically, but it would work.

I'm sure there are a lot of arguments against doing it this way and am interested in hearing them.