Each of these schemes requires some amount of knowledge that may not be determinable by examining just the data itself.

Just to make sure. I didn't imply anywhere, that the developer should determine the encoding by examining the data. Personally I believe that guessing the encoding is a sin. It should be done only if there's no other choice. It is much better to force the user to provide the information about the encoding if it is not known already.