Reducing the colorspace is a nontrivial operation that involves approximating a large set of colors with a smaller set, so it makes sense that you cannot just set number of colors. To reduce the size of the color space, check out the Posterize and Quantize methods in Image::Magick.