fvgestel has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

I've stumbled upon a weird problem in perl and I was hoping that some perl-guru here could come with an answer.
The problem is this :
2 years ago I wrote a perl-script which uses the pack function like this :

my $msg = pack("C*",0x80,0x10, 0x29)

now $msg contains 0x80,0x10 and 0x29

I'm running the same code on a redhat enterprise linux ws3 machine with Intel Xeon, only now $msg contains :

0xC2,0x80,0x10 and 0x29

It seems that every byte bigger than 0x7F gets a prefix byte! (0xC2 or 0xC3)
What's happening here?

Replies are listed 'Best First'.
Re: perl pack function
by jmcnamara (Monsignor) on Oct 26, 2005 at 13:42 UTC

    The string is being converted to UTF8.

    Since you are on Red Hat it is probably due to your $LANG environmental variable being set to a UTF8 dialect, which perl then picks up.

    --
    John.

[reply]
      Yes, LANG is set to en_US.UTF-8

      But I just tried your binmode suggestion on socket S and it seems to work!

      Thanks a lot!
[reply]
Re: perl pack function
by bart (Canon) on Oct 26, 2005 at 13:29 UTC
    It does appear to me that that string got converted to UTF8. The question is: why? Did you append that string to some UTF8 string? In such a case, perl would "promote" the Latin-1 string to UTF8, with a result as you describe.

    But perhaps this isn't really the contents of $msg. Perhaps it is only converted to UTF8 when you print it out to a file. In which case, perl thinks that's what you want. Perhaps tell it that the filehandle should be treated as raw, hence, no conversion of any characters? binmode could offer you a solution, then.

[reply]
      I'm not printing to file; I'm filling a network-packet.
      here's a more complete snippet :

              my $msg = pack("C*",0x80,0x10,0x29,
         0x00,0x00,0x01,0x00,0x00,0x00,0x00,0x00,0x01,0x20,
         @data,
         0x00,0x00,0x20,0x00,0x01,0xc0,0x0c,0x00,0x20,
         0x00,0x01,0x00,0x04,0x93,0xe0,0x00,0x06,0x60,
         0x00,@ip);
        my $len_msg = length($msg);
        my $num_short = $len_msg / 2;

        bind(S, $us);
        send(S, $msg, 0, $them) or die "send : $!";
        my $res='';
        recv(S,$res,0,0);
        close S;

      [download]
[reply]
[d/l]
Re: perl pack function
by BrowserUk (Patriarch) on Oct 26, 2005 at 13:39 UTC

    Which version of Perl are you using?

    Try 

    my $msg = pack("C0C*",0x80,0x10, 0x29)

    [download]

    It shouldn't make a difference, but I seem to remember it did one one of the early 5.8.x versions of perl.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
Re: perl pack function
by fvgestel (Initiate) on Oct 26, 2005 at 13:30 UTC
    BTW, I already tried to use pragma's :

    use bytes;
    no utf8;

    But this doesn't change anything....
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom