I'm not really sure what the Imager module documentation is saying about to_palletted(), but it does say that it may ignore those colors. Perhaps you should explore the logging methods to see if something is failing?