in reply to UTF8 - Same script, different behaviour

Your example script will never give such an error since you never tell Perl that any of your data might be anything other than Latin-1 bytes. So Perl reads and writes the data as Latin-1 bytes and sees no reason to warn.

You can only get this warning when you use Encode::decode explicitly or when you (or a module you use) decode your data from a character set encoding to Unicode implicitly, for example by

open my $fh, '<:encoding(UTF-8)', $filename or die "Couldn't open '$filename': $!";

Maybe Perl 5.24 warns in more situations than Perl 5.16 did when writing Unicode strings to a file (or somewhere else).

To correct the situation Perl warns about, be explicit in your output encoding and open your output files as

open my $fh, '>:encoding(UTF-8)', $filename or die "Couldn't open '$filename': $!";

... or encode your strings to bytes before writing them:

use charnames ':full'; use Encode 'encode'; open my $fh, '>', $filename or die "Couldn't create '$filename': $!"; my $output = "MOT\N{CAPITAL LETTER O WITH DIAERESIS}RHEAD"; my $output_bytes = encode( 'UTF-8', $output ); print $fh $output_bytes;

Replies are listed 'Best First'.
Re^2: UTF8 - Same script, different behaviour
by Melly (Chaplain) on Apr 28, 2020 at 12:27 UTC

    Ah - got it (I think)

    My actual script reads the CSV via CSV_XS, with 'binary' on, so I'm guessing that's what alerts Perl.

    Many thanks.

    UPDATE!

    Okay, so I altered my test-script to explicitly open utf-8, and now the test-script throws an error at both locations, although my main script continues to only throw an error locally - go figure.

    In any event, opening my output file with ':encoding(UTF-8)' has fixed the issue.

    map{$a=1-$_/10;map{$d=$a;$e=$b=$_/20-2;map{($d,$e)=(2*$d*$e+$a,$e**2 -$d**2+$b);$c=$d**2+$e**2>4?$d=8:_}1..50;print$c}0..59;print$/}0..20
    Tom Melly, pm (at) cursingmaggot (stop) co (stop) uk