By printing this out in ASCII hex, you have actually changed the problem.
When you do a binmode read to a scalar, you get a $buffer where each "character" is a byte (0-255 unsigned). For actual bytes there is "nothing to be done" - it is already a byte value. For multi-byte fields some sort of unpack() is usually necessary. Use substr() to get a range of byte values. Use unpack() to convert sequences of bytes into some other representation (from little endian to big endian or whatever).
my $HeaderRecordId = substr($buf,0,1);
my $FileFormatVersion = substr($buf,1,1);
my $TimeStamp = substr($buf,2,8); #some kind of unpack needed here!
my $NumBSC = substr($buf,10,1);
$HeaderRecordId $FileFormatVersion, $NumBSC are just bytes and nothing more is needed past substr().
Update: I looked back an some ancient code (I don't deal with binary very often), but this had to do with .WAV files. My point is that substr() will get you the sequence of bytes. Here, I look for "RIFF" and "data" with string compares. The V4 unpack is for little endian conversion.
code snippet...
read(IN, my $buff, 1 * 2**10);
(substr($buff,0,4) eq "RIFF") || die "not a valid RIFF file";
my $size = unpack ("V4",substr($buff,4,4));
myprint (" RIFF segment size = $size");
(substr($buff,50,4) eq "data")|| die "data segment not found";
my $dsize = unpack ("V4",substr($buff,54,4));
myprint (" DATA Segment size = $dsize");