Re: Perl seems to mistreat "PerlIO" environment variable (Windows)

I updated your code to the following, and added the output I got as comments after every command (Win 10, Perl 5.28).

@echo off
set PERLIO=
perl -E "print STDERR PerlIO::get_layers(STDOUT),qq(\n); print qq(\n)"
+ >> tmp
REM unixcrlf 0d0a
perl -E "binmode STDOUT; print STDERR PerlIO::get_layers(STDOUT),qq(\n
+); print qq(\n)" >> tmp
REM unixcrlf 0a
set PERLIO=:raw
perl -E "print STDERR PerlIO::get_layers(STDOUT),qq(\n); print qq(\n)"
+ >> tmp
REM unix 0d0a
perl -E "binmode STDOUT; print STDERR PerlIO::get_layers(STDOUT),qq(\n
+); print qq(\n)" >> tmp
REM unix 0d0a
perl -E "binmode STDOUT, ':crlf'; print STDERR PerlIO::get_layers(STDO
+UT),qq(\n); print qq(\n)" >> tmp
REM unixcrlf 0d0d0a
set PERLIO=:raw:crlf:pop
perl -E "print STDERR PerlIO::get_layers(STDOUT),qq(\n); print qq(\n)"
+ >> tmp
REM unix 0a
perl -E "binmode STDOUT; print STDERR PerlIO::get_layers(STDOUT),qq(\n
+); print qq(\n)" >> tmp
REM unix 0a
perl -E "binmode STDOUT, ':crlf'; print STDERR PerlIO::get_layers(STDO
+UT),qq(\n); print qq(\n)" >> tmp
REM unixcrlf 0d0a
set PERLIO=
perl -Mopen=IO,raw -E "say unpack 'H*', $_ while <ARGV>" tmp
del tmp
[download]

In a bug I reported a while back, I got this explanation in regards to the :crlf layer on Windows:

:crlf is not just a translation layer on top of the buffering system, it's an alternative buffering system that will optionally do the translation. Calling binmode will disable that translation but not the layer. For extra confusion, on Linux binmode will pop off the crlf layer, because it's not presumed to be the only buffering layer.

What I think this means is that anytime the :crlf layer is missing on Windows, the behavior may be wonky. This appears to be triggered by PERLIO=:raw, and may very well be a bug. Is there any way you can avoid that environment variable being set and only use binmode?

Comment on Re: Perl seems to mistreat "PerlIO" environment variable (Windows) Select or Download Code

Replies are listed 'Best First'.
Re^2: Perl seems to mistreat "PerlIO" environment variable (Windows) by vr (Curate) on Apr 03, 2020 at 12:23 UTC
Is there any way you can avoid that environment variable being set and only use binmode? Yes, sure, it's just that it's distribution of `App::*` variety, end-user targeted, like "no serviceable parts inside", etc. Of course I can fix it (and should notify the author). Thanks for reminding about `crlf` being "alternative buffering" and therefore operating on raw bytes: `open my $fh, '>', 'tmp.txt'; binmode $fh, ':encoding(UTF16)'; print $fh qq(\N{U+010A}\n); close $fh;` [download] output is invalid on Windows (fe ff 01 0d 0a 00 0d 0a). Should have used `':raw:encoding(UTF16)'`. Oh, as I see prepending `':raw'` regardless is a good practice since long ago, should have paid closer attention to such details.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: Perl seems to mistreat "PerlIO" environment variable (Windows)
by vr (Curate) on Apr 03, 2020 at 12:23 UTC

Is there any way you can avoid that environment variable being set and only use binmode?

Yes, sure, it's just that it's distribution of App::* variety, end-user targeted, like "no serviceable parts inside", etc. Of course I can fix it (and should notify the author). Thanks for reminding about crlf being "alternative buffering" and therefore operating on raw bytes:

open my $fh, '>', 'tmp.txt';
binmode $fh, ':encoding(UTF16)';
print $fh qq(\N{U+010A}\n);
close $fh;
[download]

output is invalid on Windows (fe ff 01 0d 0a 00 0d 0a). Should have used ':raw:encoding(UTF16)'. Oh, as I see prepending ':raw' regardless is a good practice since long ago, should have paid closer attention to such details.

[reply]
[d/l]
[select]