in reply to Re^15: Seeking Perl docs about how UTF8 flag propagates (Terminology)
in thread Seeking Perl docs about how UTF8 flag propagates

Something to keep in mind is that the fact that a term can mean two different things doesn't necessarily mean it's a confusing term. If only one of each definition is used in a given context, then there's no confusion. For example, it would be perfectly fine to say that pack( "N", 200 ) returns a string of 4 bytes. Doesn't matter how many bytes of storage is used for it. It's still the same string of 4 bytes if you upgrade the resulting scalar. Because we're not talking about how it's stored. We're talking about how it would be stored in a stream. So yeah, the string of 4 bytes might use 5 bytes of storage in the string buffer. And the same the string of 4 bytes takes up 42 bytes of memory (according to Devel::Size). Yet none of that is confusing.

a character is encoded by 1-4 characters ... WTF? )

Would you say "I put the vehicle on the vehicle", or would you say "I drove my car onto the ferry"?

The fact that the process involves string operations and thus characters isn't relevant. You always have the option of being more specific (e.g. using Code Point of character, and byte instead of character) if it makes things clearer.

That's why one would say a Code Point encodes to one to four bytes.

with it's one to four bytes.

Each character takes up 1 to 13 bytes of storage, actually.

As you pointed out are most (not all) string operators in Perl "character" based.

I did not sat that. Quite the opposite. All string operations deal with characters. By definition. A string is made up of characters. I literally said that's the name of the elements of a string.

If a terminology invites for misunderstandings one should chose a new word.

I welcome an unambiguous word for "string element". But until one comes around, I shall continue defining the terminology I use, and part of that is defining a character to be a string element. Cause no one wants me to say string elements.

But did you notice that the terminology I said I use didn't mention "character" at all? I think you're trying to convince me that "character" is confusing, yet I didn't even use the word! It's a term that can usually be avoided entirely. It only comes into play when dealing with strings of arbitrary content. There's usually no reason for a term for string elements otherwise.

  • Comment on Re^16: Seeking Perl docs about how UTF8 flag propagates (Terminology)
  • Download Code

Replies are listed 'Best First'.
Re^17: Seeking Perl docs about how UTF8 flag propagates (Terminology)
by LanX (Saint) on May 22, 2023 at 11:29 UTC
    > Each character takes up 1 to 13 bytes of storage, actually.

    I'm puzzled, could you please give an example of a string where a character takes more than 4 bytes of storage?

    edit

    > I think you're trying to convince me that "character" is confusing,

    no, I'm not,

    as I said I think byte := character is confusing, it's rather character := byte in the classic case.

    (Upgrade: deleted part about using vec and unpack with wide characters, until further testing)

    Cheers Rolf
    (addicted to the 𐍀𐌴𐍂𐌻 Programming Language :)
    Wikisyntax for the Monastery

[reply]
[d/l]
[select]

      Anything larger than 0x1F_FFFF will take more than four bytes.

      Anything larger than 0xF_FFFF_FFFF will take 13. 

      $ perl -MDevel::Peek -e'Dump(chr(0x7FFFFFFFFFFFFFFF))'
SV = PV(0x5555770a3f00) at 0x5555770d2d50
  REFCNT = 1
  FLAGS = (PADTMP,POK,READONLY,PROTECT,pPOK,UTF8)
  PV = 0x5555770db6f0 "\xFF\x80\x87\xBF\xBF\xBF\xBF\xBF\xBF\xBF\xBF\xB
+F\xBF"\0 [UTF8 "\x{7fffffffffffffff}"]
  CUR = 13
  LEN = 15

      [download]

      as I said I think byte := character is confusing, it's rather character := byte in the classic case.

      Still no idea what you mean. Are you talking about something I said? In which case, you could point out which thing I said you find confusing?

      oh, and the classic case is char ≡ byte ≡ smallest addressable unit. It has been all kinds of sizes, including 6, 8, 9, 16 and even 32 bits. (I personally worked on a system with a 16 bit byte extensively.) But we now have strings with elements that have a size larger than the smallest addressable unit. So the classic case is irrelevant.

[reply]
[d/l]
[select]
[reply]

In Section Seekers of Perl Wisdom