Perl has two kinds of strings. Strings of bytes and strings of characters. When you use use utf8, literals are decoded from a string of UTF-8 bytes into a string characters.

use Encode qw( decode );

$s = "高";
print(length($s), "\n");  # 3

$s = decode('utf8', "高");
print(length($s), "\n");  # 1

$s = do { use utf8; "高" };
print(length($s), "\n");  # 1
[download]

→ Replace 高 with the UTF-8 encoding of 高 in the above source.

Writting to a ':utf8' handle reverses the process.

use Encode qw( encode );

my $s = do { use utf8; "高" };
print(length($s), "\n");   # 1

open my $fh, '>:utf8', \$s2;
print $fh $s;
print(length($s2), "\n");  # 3

$s2 = encode('utf8', $s);
print(length($s2), "\n");  # 3
[download]

→ Replace 高 with the UTF-8 encoding of 高 in the above source.

In context that means $str contains 6 characters, while $output2 contains 18 bytes. Since both contain wildly different contents, it's to be expected that they behave differently.

The warning is given when a string of characters is outputed in :bytes mode (the default, the opposite of :utf8). The gibberish comes from encoding a string twice. One solution:

use strict;
use warnings;
use utf8;

use Encode qw( decode );

my $chars_src = "采样速率太高";
my $bytes_dst;

{
   open my $fh, '>:utf8', \$bytes_dst;
   print $fh $chars_src;
}

my $chars_dst = decode('utf8', $bytes_dst);

binmode(STDOUT, ':utf8');
print("$chars_src\n");
print("$chars_dst\n");
[download]

→ Replace 采样速率太&#39640 with the UTF-8 encoding of 采样速率太高 in the above source.

Another solution:

...

my $bytes_src = encode('utf8', $chars_src);

print("$bytes_src\n");
print("$bytes_dst\n");
[download]

Another solution:

...

binmode(STDOUT, ':utf8');
print("$chars_src\n");
binmode(STDOUT, ':bytes');
print("$bytes_dst\n");
[download]

I like using hungarian notation (bytes_ vs chars_) when dealing with encoded and unencoded values. Making Wrong Code Look Wrong is a good essay on the subject.

Update: Added extra solutions.
Update: Added "reverses the process" bit.

(update: you should do binmode STDOUT, ":utf8"; as well, if you're going to print wide characters to STDOUT.)

Another update: (sorry -- I should have read the whole post, carefully, sooner) So you are printing utf8 data to an in-memory scalar-variable-as-file, and then you want to just print the contents of that scalar as if it were not an in-memory file (but just a utf8 string).

The problem there is that using the scalar as a file makes it impossible for perl to realize that it's really a utf8 string (the fact that you "opened" the file with ">:utf8" does not cause the variable's "utf8 flag" to be turned on).

So if you want perl to treat that variable as a utf8 string, you have to set the utf8 flag, and the supported way to do that is to use Encode::decode...

use Encode;

# ... treat $output2 as an output file, then:

$output2 = decode( "utf8", $output2 );

# assuming that STDOUT is set to utf8 mode, you can now safely

print "$output2\n";
[download]

Ah, that makes a heap of sense - thank you.

Actually putting the binmode STDOUT, ":utf8"; after the 'corrupting' print:

print "\n" . $output2;
binmode STDOUT, ":utf8";
print "\n$str";
[download]

'fixes' the problem too, but isn't near as clean as the decode solution.

---

DWIM is Perl's answer to Gödel

I don't need binmode OUTVAR2, 'utf8'; because I already opened the handle in utf8 mode (OP: "Changing the open to use '>:utf8'...").

If I add binmode STDOUT, ":utf8"; then I get the OUTVAR2 streamed text double encoded (as indicated in the OP).

---

DWIM is Perl's answer to Gödel