in reply to Re^4: Encoding Problem - UTF-8 (I think)
in thread Encoding Problem - UTF-8 (I think)

how about a meditation? ;)

Given the kneejerk responses in this thread, I think that would be pointless.

In a nutshell:

It exists; and railing against it now is pretty pointless; but I'm at a (apparently enviable) point in my career and life where I do not have to deal with this crock, so I do not.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re^5: Encoding Problem - UTF-8 (I think)

Replies are listed 'Best First'.
Re^6: Encoding Problem - UTF-8 (I think)
by Anonymous Monk on Dec 16, 2015 at 13:36 UTC
    Given the kneejerk responses in this thread, I think that would be pointless.
    Okay.

    I started to write a long reply, but this is indeed probably not the best place to discuss that... I understand your point.

    UTF-8 is a bit lame, and yet it's the best general-purpose encoding today (at least, among those that are actually used). The "space savings" probably always was the least interesting thing about it.

[reply]
      this is indeed probably not the best place to discuss that

      We could move to email if there is more to say? (But how to exchange emailids with Anonymonk?)

      and yet it's the best general-purpose encoding today

      That's a bit like saying Kim Jong-un is the best leader in NK :)

      For the web -- with what are effectively signatures embedded in the Accept-Charset header -- it works well enough, but the moment you put it into a file there is no provision for identifying the particular encoding, and you're back to guessing.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        That's a bit like saying Kim Jong-un is the best leader in NK :)
        Not at all! Well, maybe he is? :) But anyway, about UTF-8:
        1. It's easily recognizable. It's just extremely unlikely that you'll get a (not super short) string that just happens to look like valid UTF-8.

          OTOH, things like UTF-16LE, or (especially) one-byte stuff like Latin-1 "look like" complete binary garbage.

        2. Just remove a couple of random bytes from a UTF-8 string, and you'll lose a couple of characters. All others are still there, completely undamaged.

          Remove a couple of bytes in the middle of a UTF-32 string, and the rest of the string IS binary garbage.

        One byte encodings are just not general purpose... Since some users want to use all kinds of characters in their documents. Look: 

        абвгд

        [download]
        Yeah, perlmonks uses a one byte encoding... Windows-1252, I believe.

        Now, there could be a self-synchronizing, easily recognizable, fixed-length encoding, but it wouldn't be backwards compatible with 7-bit ASCII. So what did you expect? If it's not backwards, it's not compatible...
[reply]
[d/l]

In Section Seekers of Perl Wisdom