binmode is not part of the buffered IO scheme - in fact, the last bit of the perldoc for binmode explicitly says:

binmode() is not only important for readline() and print() operations, but also when using read(), seek(), sysread(), syswrite() and tell() (see the perlport manpage for more details). See the "$/" and "$\" variables in the perlvar manpage for how to manually set your input and output line-termination sequences.

Now, as far as the original post is concerned, if you are only ever worried about a single argument in ARGV, this would be sufficient:

if (! @ARGV) {
  use open ':raw'; # implicit binmode on each open
  open(ARGV, '-');
} else {
  use open ':raw';
  open(ARGV, shift @ARGV); # Yeah, it's the two-arg version,
                           # because that's what perl does with <>
                           # see perlopentut
}
# ... now go do all that sysread stuff.
[download]

If, however, there are possibly multiple files available, you'll have to get more creative:

my @files = (@ARGV)?@ARGV:qw(-);
NEXTFILE:
while (@files) {
  use open ':raw';
  open(ARGV, shift @files);
  # do sysread stuff here
  # don't forget that an eof here may not be a "real" eof,
  # but a signal to go to the next file
}
[download]

Update: Oops, as Mr. Muskrat pointed out. Fixed.

@/=map{[/./g]}qw/.h_nJ Xapou cets krht ele_ r_ra/;
map{y/X_/\n /;print}map{pop@$_}@/for@/
[download]

In your first snippet, you've reversed what needs to happen.

if (@ARGV) { # something's in @ARGV so we'll use it
  use open ':raw'; # implicit binmode on each open
  open(ARGV, shift @ARGV); # Yeah, it's the two-arg version,
                           # because that's what perl does with <>
                           # see perlopentut
} else {     # nothing's in @ARGV so we'll use STDIN
  use open ':raw'; # implicit binmode on each open
  open(ARGV, '-');
}
# ... now go do all that sysread stuff.
[download]