in reply to Question about binary file I/O

The bitwise aspects have been answered already. Re the overall structure of your program, you should use strict and warnings, lexical file handles, and 3-argument open. Also, read is rarely used or needed in Perl programs. Finally, you should set binmode on both files to indicate they are binary files. I suggest something like: 

use strict;
use warnings;

my $infile  = 'yourinputfilename';
my $outfile = 'youroutputfilename';

open(my $fhin, '<', $infile) or die "error: open '$infile': $!";
binmode($fhin);
open(my $fhout, '>', $outfile) or die "error: open '$outfile': $!";
binmode($fhout);

local $/ = \2;   # make <> operator read two bytes at a time
while (my $TileData = <$fhin>)
{
   if (length($TileData) != 2)
   {
      warn "oops, file is of uneven length, last byte='$TileData'\n";
      last;
   }
   # do your thing with bitwise stuff here...
   # my $NewTileData = ...
   print {$fhout} $NewTileData;
}

[download]

Replies are listed 'Best First'.
Re^2: Question about binary file I/O
by TheMartianGeek (Acolyte) on Feb 06, 2011 at 02:38 UTC
    BrowserUk: Hm, I thought that might be the problem, that read() was returning a string instead of a number (and it seemed to be the case while debugging). But I looked all over for a function to convert string values to (decimal) numbers and couldn't find one.

    james2vegas: Don't pack and unpack operate only on strings and lists, though?

    eyepopslikeamosquito: I did set binmode on both files...and I had "use strict" in the program originally, but it seemed to do nothing but cause trouble; it mostly meant that I had to put "my" before almost every variable, so I just got rid of it. I'm also not sure what you mean by lexical file handles and 3-argument open. If read is rarely used or needed, though, so much the better; it seems rather rigid and cumbersome. What exactly does $/ do, though? It is defined as "the input record separator", so...does setting it to \number simply tell file handles and the like to return number characters at a time?
[reply]

      I had "use strict" in the program originally, but it seemed to do nothing but cause trouble; it mostly meant that I had to put "my" before almost every variable, so I just got rid of it. I'm also not sure what you mean by lexical file handles and 3-argument open.
      The "my" is actually very useful because, apart from catching typos in misspelt variable names, it limits the scope of the variable. A variable declared with "my" is known as a lexical variable because it has lexical scope; that is, its name is known from the point of declaration to the end of the block it is declared in (or end of file if not declared in a block). Hence my recommendation to use lexical file handles (e.g. my $fhin) rather than your global variable bareword file handle IN.

      Using lexical file handles is better style because:

      • They are local variables and so avoid the generally evil programming practice of keeping state in global variables.
      • They close themselves automatically when their lexical variable goes out of scope.
      • They avoid the confusion of barewords. IMHO, barewords are an unfortunate Perl feature and should be avoided (except when writing poetry).

      As for why the three-argument form of open is preferred, note that the old two-argument form of open is subject to various security exploits as described at Opening files securely in Perl.

      See also the first four items of Perl Best Practices Chapter 10 (I/O), namely:

      • Don't use bareword filehandles
      • Use indirect filehandles
      • If you have to use a package filehandle, localize it first
      • Use either the IO::File module or the three-argument form of open

[reply]
[d/l]
[select]
        Well, using this seemed to work:

        my $NewTileData = pack 'v', ((unpack 'v',$TileData) & 0xFC7F) | $GFXFileBits;

        Perhaps I was just misunderstanding the pack and unpack functions? The way perldoc words the explanation of them suggest that pack always returns a string and unpack always returns a list of values, neither of which I would want in this context. Thank you all for the help, in any case. (I am still confused about how pack and unpack work and what they return depending on the template, though...)
[reply]
[d/l]
        Okay, I fixed all the scope issues and stuff. It still doesn't work, though: 
        {
    local $/ = \2;
    my $TileData = 0x0000;
    
    while(my $TileData = <$fhin>)
    {
        if(length($TileData) != 2)
        {
            warn "Warning: The input file had an uneven length.  Last 
+byte read: $TileData.";
            last;
        }
        my $NewTileData = (($TileData & 0xFC7F) | ($GFXFileBits));
        print {$fhout} $NewTileData;
    }
}

        [download]
        $TileData is still a string instead of a number, so it doesn't work with the bitwise operators.
[reply]
[d/l]
        Okay, I have a new file-related problem...what's the best way to read a certain number of bytes (say, 12) beginning at a certain position in a file (say, position 40), possibly reading them into an array and possibly performing operations on them, then write the result to a certain position in another file. I thought I had something, but I didn't...I don't even see a function in Perl for overwriting a string of bytes in a file, let alone know how to do the rest.
[reply]
      You are giving pack a list, it just happens to have only one element, and you are getting a string back, you can replace 'S*' with 'S1' to make it clearer.

      Update: the 'trouble' strict mode causes is to ensure you declare your variables, preventing typoed variables from being accepted without complaint, for example. As to $/, reading in perlvar documents its behaviour when set to a reference to an integer, or variable containing an integer 'will attempt to read records instead of lines, with the maximum record size being the referenced integer. '. As to read and the like, there are a lot less cumbersome versions in IO::Handle which lets you treat filehandles like objects with methods.
[reply]
[d/l]

In Section Seekers of Perl Wisdom