in reply to Re^20: Interleaving bytes in a string quickly
in thread Interleaving bytes in a string quickly

...can silently encode your bytes using UTF-8.", .... Any encoding has to be done, explicitly, by the programmer.

No, it doesn't.

Yes. It does!

I even gave an example.

No. You did not! And you still have not--given an example of encoding occuring "silently".

Perl does not assign meaning to the values inside the string*.

What is PV w/ UTF8=0 or a PV w/ UTF8=1., if it is not "Perl ... assigning a meaning"?

At the C-level, there is no difference; but at the Perl level there most definitely is. And it is at the C-level I am calling SvPVX(). For the very reason I do not want to make any such distinction. The same would not be true if I used SvPVBytes() as you suggested.

If you guarantee that you give a string in UTF8=0 format (say by calling utf8::downgrade before calling interleave) you won't have a problem.

You are still getting it arse backward. I don't need to call downgrade(), because I know I'm never going to call upgrade(), or anything else that might cause perl to assign any other meaning than bytes to my data.

That hadn't been specified until now.

Oh, but it has. Over and over:

  1. I could use it. I know the data I'd be passing are byte strings. Nothing else makes sense given the purpose of the code.
  2. But given I'm reading the file (binary data) in raw mode, how would they get "silently encoded"?
  3. Data either originates from within my program, or from without. And in either case, Perl will treat it as bytes unless I do something explicit to indicate that it should do otherwise. And since I know I'm not going to do that, I do not have to consider it.
Your games aren't fun.

I'm not playing games.. Far from it.

You asserted: "It can silently encode your bytes using UTF-8.". Since you have demonstrably more knowledge of unicode than I, despite my strong belief that this was impossible, I was unsure enough to ask: Okay. How could it "silently encode my bytes as utf"?. Because if it was true, I wanted to know how.

But, despite your protestations to the contrary above, you have still not provided an example of how this can happen. I now know this is because you cannot do so. Because it is impossible.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"I'd rather go naked than blow up my ass"

Replies are listed 'Best First'.
Re^22: Interleaving bytes in a string quickly
by ikegami (Patriarch) on Mar 01, 2010 at 13:57 UTC

    What is PV w/ UTF8=0 or a PV w/ UTF8=1., if it is not "Perl ... assigning a meaning"? 

    uint8  i = 2;
uint16 j = i;

    [download]

    Does the second assignment change the meaning of 2? No. It's still the number of cats my friends has. 

    my $i = "\x82";
utf8::upgrade( my $j = $i );

    [download]

    Does the second assignment change the meaning of "\x82"? No. It's still the HDMI code for something or other.

    because I know I'm never going to call upgrade(), or anything else that might cause perl to assign any other meaning than bytes to my data.

    Presumably you mean "anything else that might cause perl to use the 32/64-bit string format", then why didn't you say so?

    It's a very weird assumption to make, but you're free to make all the assumptions you want.

[reply]
[d/l]
[select]
      uint8  i = 2; uint16 j = i; Does the second assignment change the meaning of 2?

      Cheap theatrics. Cos now int i = -1; uint j = i; it does.

      Presumably you mean "anything else that might cause perl to use the 32/64-bit string format",

      Nope! I meant exactly what I said.

      Paraphrase: If I populate the PV of a perl scalar with byte values, and later call SvPVX() to gain access to those byte values, I will get back exactly what I put in them...unless in the interim, I explicitly do something to change them. No presumptions or assumptions. Simply fact.

      And using SvPVX() will NEVER cause it to

      silently encode your bytes using UTF-8.
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]

        Cheap theatrics. Cos now int i = -1; uint j = i; it does.

        I never said or implied that the types are equivalent. Different types have different range, precision and costs.

        32/64-bit strings have a wider range, but they are more costly in every respect. (That's why we have 8-bit strings at all.) The range of 32/64-bit strings encompasses the range of 8-bit strings, so your signed vs unsigned analogy is broken.

        Nope! I meant exactly what I said.

        If you meant what you said, you haven't ruled out doing something like $pascal_bytes = chr(length($bytes)) . $bytes;. For long enough strings, it upgrades without calling upgrade and without Perl assigning any meaning to the string.

[reply]
[d/l]

In Section Seekers of Perl Wisdom