in reply to Re: binmode copy loses final byte
in thread binmode copy loses final byte

#!/usr/bin/perl use strict; use warnings; my $in_qfn = '...'; my $out_qfn = '...'; open( my $in_fh, "<:raw", $in_qfn ) or die( "Can't open `$in_qfn`: $!\n" ); open( my $out_fh, ">:raw", $out_qfn ) or die( "Can't create`$out_qfn `: $!\n" ); my $int_size = 4; # length( pack( "L<", 0 ) ); local $/ = \( 8 * 1024 ); while ( my $in_buf = <$in_fh> ) { length( $in_buf ) % $int_size == 0 or die( "Invalid input file `$in_qfn`\n" ); my @ints = unpack( "L<*", $in_buf ); # Something that modifies `@ints` here. print( $out_fh pack( "L<*", @ints ) ) or die( "Error writing to `$out_qfn`: $!\n" ); } close( $in_fh ) or die( "Error reading from `$in_qfn`: $!\n" ); close( $out_fh ) or die( "Error writing to `$out_qfn`: $!\n" );

Without readline (<>) and print:

#!/usr/bin/perl use strict; use warnings; my $in_qfn = '...'; my $out_qfn = '...'; open( my $in_fh, "<:raw", $in_qfn ) or die( "Can't open `$in_qfn`: $!\n" ); open( my $out_fh, ">:raw", $out_qfn ) or die( "Can't create`$out_qfn `: $!\n" ); my $blk_size = ( stat( $in_fh ) )[ 11 ] || 16384; my $int_size = 4; # length( pack( "L<", 0 ) ); $blk_size % $int_size == 0 or die( "Invalid block size\n" ); my $in_buf = ''; while ( 1 ) { my $bytes_read = sysread( $in_fh, $in_buf, $blk_size, length( $in_b +uf ) ); defined( $bytes_read ) or die( "Error reading from `$in_qfn`: $!\n" ) ; $bytes_read or last; my @ints = unpack( "L<*", $in_buf ); substr( $in_buf, 0, @ints * $int_size, "" ); # Something that modifies `@ints` here. { my $out_buf = pack( "L<*", @ints ); my $total_to_write = length( $out_buf ); my $total_written = 0; while ( $total_to_write ) { my $bytes_written = syswrite( $out_fh, $out_buf, $blk_size, $ +total_written ); defined( $bytes_written ) or die( "Error writing to `$out_qfn`: $!\n" ); $total_written += $bytes_written; $total_to_write -= $bytes_written; } } } length( $in_buf ) == 0 or die( "Invalid input file `$in_qfn`\n" ); close( $in_fh ) or die( "Error reading from `$in_qfn`: $!\n" ); close( $out_fh ) or die( "Error writing to `$out_qfn`: $!\n" );

Replies are listed 'Best First'.
Re^3: binmode copy loses final byte
by aplonis (Pilgrim) on Jul 01, 2025 at 00:46 UTC

    Both of those examples die, giving the 'Invalid input file...' message part way through reading an input file.

    An output file is written, but only up to the point where reading died.

    Happens both with a very large *.txt file, and also a medium sized *.jpg file. Nothing looks untoward at that point for the *.txt input file. The output file ends in the middle of a text line.

      Then give it a valid input file! Your program needs a file that consists of a sequence of "packed" int objects, but you didn't provide such a file. Put differently, you need a file whose size is a multiple of sizeof(int).

      Or, from the other perspective, you wrote a program that needs a file whose size is a multiple of sizeof(int), but that's not what the program should expect.

      When using a block ciphers on a stream of arbitrary length, a padding algorithm is used to make the input size a multiple of the block size before encryption.

      In your case, you could use the following:

      my $padding_len = $int_size - ( length( $buf ) % $int_size ); my $padding = ( "\x00" x ( $padding_len - 1 ) ) . chr( $padding_len ); $buf .= $padding;

      After decrypting, you remove the padding by removing an amount of bytes equal to the value of the last bytes.

      $buf = substr( $buf, 0, -ord( substr( $buf, -1 ) ) );

        Yes. That was the issue. Thank you kindly. An encryption program, naturally, must expect to read in files of whatever type.

        What I'm hoping to do is recreate in Perl an encryption program which I had written long decades past and subsequently lost. The original was among many which I had authored in JForth on an Amiga 2000. Those I had preserved for ten years or more on a CDROM but now cannot find that disk anywhere.

        I have all but entirely forgotten Forth, and my Perl has also grown rusty. Being now retired, I have elected to refresh myself in both as hobby projects. Again, I thank you for your indulgence.