use utf8;
sub fileread;  # use do { local $/; <$fh> }
my $file = 'test_utf8';

# Test 1
binmode STDOUT, ':encoding(UTF-8)';
my $line = fileread $file,':raw';
utf8::decode($line);

if ($line =~ /(&#10055;)/) { print "found '$1'\n"; }
print $line;


sub fileread {
  my ($file,$enc) = @_;
  my $string; my $stref = \$string;
  open(my $fh, "< $enc", $file) || die "Can't open $file: $!";
  ${$stref} = do { local $/; <$fh> };
  return $string;
}
[download]

my $line = fileread $file,':encoding(UTF-8)';
$line = Encode::decode('UTF-8', $line, 'Encode::FB_CROAK');
[download]

ps I notice all the occurrences of ❇ in my code blocks have been turned into ❇ so it's some small comfort that perlmonks.org can't quite get a grip on this either. 😜

Really glad to see you now have a working SSCCE. I'm somewhat bemused by the use of scalar ref for the string in the subroutine - can you elaborate on why that is desired or necessary?

There should be no problem with using Encode::decode() to do the decoding, and that is often what I use. Since you put the effort in to provide your SSCCE, here is mine in return, using this sub.

use strict;
use warnings;
use utf8;

use Encode qw/decode/; # For explicit decoding only

binmode STDOUT, ':encoding(UTF-8)';

my $file = 'test_utf8';

print "Explicitly decoded:\n";
my $encoded_text = fileread ($file, ':raw');
my $text = decode ('UTF-8', $encoded_text, Encode::FB_CROAK);

if ($text =~ /(❇)/) { print "found '$1'\n"; }
print $text;

print "\n\nImplicitly decoded:\n";
$text = fileread ($file, ':encoding(UTF-8)');

if ($text =~ /(❇)/) { print "found '$1'\n"; }
print $text;

sub fileread {
  my ($file,$enc) = @_;
  open my $fh, "< $enc", $file or die "Can't open $file: $!";
  local $/;
  my $string = <$fh>;
  close $fh;
  return $string;
}

As is hopefully clear, this shows that the same result occurs whether by having the PerlIO layer perform the decoding implicitly or by performing it explicitly with Encode::decode() in the code (as you would need to do in order to process your FCGI parameters, for example). I have simplified fileread() to remove the apparently unnecessary scalar ref too.

I notice all the occurrences of ❇ in my code blocks have been turned into ❇

Yes, it is a known issue and is an unfortunate consequence of this site pre-dating much of unicode handling. If you have utf-8 characters in your source you can use <pre> tags as I have here.

---

🦛

Really glad to see you now have a working SSCCE. I'm somewhat bemused by the use of scalar ref for the string in the subroutine - can you elaborate on why that is desired or necessary?

Like most things Perl it's an embarrassing historical artifact, I cut'n'pasted from my usual file slurping routine which at a sprightly 50 lines was too big to fit in the SSCCE and didn't really think about it. Speaking of not thinking things through is there anything really wrong with the following SSCCE which accomplishes my original goal of bypassing perlio completely and using _utf8_on? Apart from the fact taint doesn't like it it seems like it would really speed up working on files that I know are utf-8.

use utf8;
use Encode qw(_utf8_on);
my $file = 'test_utf8';

binmode STDOUT, ':encoding(UTF-8)';
my $line = &sysfileread($file);
_utf8_on($line);

if ($line =~ /(❇)/) { print "found '$1'\n"; }
print $line;

sub sysfileread {
  my ($file) = @_;
  my $string;
  open(my $fh, "<", $file) || die "Can't open $file for reading: $!";
  my $size_left = -s $fh;
  while( $size_left > 0 ) {
    my $read_cnt = sysread($fh, $string, $size_left, length $string);
    last unless( $read_cnt );
    $size_left -= $read_cnt;
  }
  return $string;
}

I notice all the occurrences of ❇ in my code blocks have been turned into &#10055;

For more emoji fun, I like to make my PM snowflakes colourful - for example, from ❄ (&#10052;) to ❄️ (&#10052;&#65039;) to prettify this Snow flake obfu. :-)

👁️🍾👍🦟