This is perl, v5.8.6 built for darwin-thread-multi-2level
[download]

open (FH, '<:raw:perlio', '/dev/null');
print(PerlIO::get_layers(FH));          # --> unix perlio
[download]

$ perl --version

This is perl, v5.8.8 built for i486-linux-gnu-thread-multi
...

$ perl -MPerlIO -e 'open($fh, "<", "/dev/null");
print join(", ", PerlIO::get_layers($fh)), "\n";'
unix, perlio
$ perl -MPerlIO -e 'open($fh, "<:raw", "/dev/null");
print join(", ", PerlIO::get_layers($fh)), "\n";'
unix
$ perl -MPerlIO -e 'open($fh, "<", "/dev/null");
binmode $fh;
print join(", ", PerlIO::get_layers($fh)), "\n";'
unix, perlio
[download]

For what it's worth, I tend to use the binmode $fh; variant, but that's a habit formed in pre-layer times.

Three data points (Linux, OS X, and Win32) that do little to clarify the confusion.

Summary:

5.8.5 linux  <                    => unix perlio
5.8.5 linux  <            binmode => unix perlio
5.8.5 linux  <:raw                => unix 
5.8.5 linux  <:raw:perlio         => unix perlio

5.8.6 darwin <                    =>             stdio
5.8.6 darwin <            binmode =>             stdio
5.8.6 darwin <:raw                => unix
5.8.6 darwin <:raw:perlio         => unix perlio

5.8.6 win32  <                    => unix        crlf
5.8.6 win32  <            binmode => unix        crlf
5.8.6 win32  <:raw                => unix 
5.8.6 win32  <:raw:perlio         => unix perlio
[download]

Detail:

First of all, thank you very much for all your responses !

As jbert pointed out in his reply this seems to be either a bug in PerlIO itself or an erroneous (or at least misleading) documentation.

So my conclusion is:

To open() a file containing binary data in a portable manner without loosing buffering use the sequence

  open(my $fh, '<', $filename);
  binmode($fh);
[download]

and avoid

  open(my $fh, '<:raw', $filename)
[download]

how tempting it may look at first.

Some additional comments to my original post:

Depending on what you want to do with your 'raw' filehandle, then using:

 open(my $fh, '<', $filename);
 binmode($fh);
[download]

...may not be appropriate (at least on windows).

If you want to read raw bytes from a filehandle (for example to look for a BOM) and _then_ add an encoding layer then you need to ensure that there isn't a :crlf layer hanging around to begin with which the above has (it may not be enabled, but it's still there *). This is because of :crlf not working nicely with encodings eg. it breaks tell().

So, the method i use is:

open(my $fh, "<:raw:perlio", $file);

# read first few bytes for a BOM or ASCII in utf(16|32)
#  to determine encoding

binmode $fh, ":encoding($encoding)"
[download]

* Consider:

open $fh, "<", $^X or die;
binmode $fh;
print "1: ", join(' ', PerlIO::get_layers($fh, details => 1)), "\n";
close $fh;

open $fh, "<:raw:perlio", $^X or die;
print "2: ", join(' ', PerlIO::get_layers($fh, details => 1)), "\n";
close $fh;
[download]

gives:

1: unix  4195328 crlf  4195328
2: unix  4195328 perlio  4195328
[download]

Thanks for your advice !

It's interesting that binmode() only disables the :crlf layer but doesn't remove it. (Just discovered, that's actually documented in PerlIO.)

Does it still interfere with tell() even when it's disabled ? (perlport says: "If you use binmode on a file, however, you can usually seek and tell with arbitrary values in safety.")

I'm just curious, what happens if you try these on Windows:

open $fh, "<", $^X or die;
binmode $fh, ":raw";        # perldoc says: identical to 'binmode $fh'
print "3: ", join(' ', PerlIO::get_layers($fh, details => 1)), "\n";
close $fh;

open $fh, "<", $^X or die;
binmode $fh, ":raw:perlio"; # seems not useful (see below)
print "4: ", join(' ', PerlIO::get_layers($fh, details => 1)), "\n";
close $fh;
[download]

On UNIX that gives:

3: unix  4195328 perlio  4195328
4: unix  4195328 perlio  4195328 perlio  4195328
[download]

---

I know this discussion should probably go into a Perl develompent group but let me just explain where I'm having problems to understand things here:

If one has to use open($fh, '<:raw:perlio', $file) to get a clean and buffered binary stream this unconditionally forces a :perlio layer onto the stack, which in turn inserts a :unix layer below it.

Now, what about MacOS where Perl uses :stdio by default ? (I know far too little about MacOS but it must be faster than :unix:perlio there.) Also if someone wants to change the default layers via the PERLIO environment variable (for whatever reason), it would be blithely ignored on those streams.

Thanks for testing !

Yes, if you explicitly specify the :perlio (or on MacOS maybe better :stdio) layer the stream will be buffered. What I don't understand (yet ?) is why a simple :raw removes all layers, including the existing :perlio (or in your case :stdio) layer.

But I have the feeling, that that behaviour may be intentional and that only the documentation is a bit misleading here.