Bottom line: will my approach of < 32 ascii or > 126 ascii work despite the actual encoding sent?

Not reliably. There are character encodings like UTF-7 that don't fit that scheme.

It's really better to determine the encoding first (maybe with Encode::Guess (core module)), and then properly decode it with Encode::decode.