jmlynesjr has asked for the wisdom of the Perl Monks concerning the following question:

SOLVED: Trial and error with pack options.

my $c = pack('W*', @binbytes);
print $out $c;

[download]

Been away for awhile...

Background: I recently discovered the Simh/Open-Simh open source project. This project provides a framework for building simulators for vintage computers(many minis are currently supported - DG, DEC, HP, Honeywell, XDS, etc.). As a Data General(DG) Nova/Eclipse programmer from the 70s, I decided to download the Nova simulator and give it a try. It's a very slick piece of software. It runs the original Data General binary software distributions at about 40x speed on my Dell laptop. I compiled and ran a FORTRAN IV code from the 70s. The simulator represents peripheral devices as unstructured linux files(Ubuntu 22.04 LTS in my case). I want to be able to move text files from the DG RDOS filesystem to/from the linux file system mostly to have access to an editor that I remember how to use.

I'm trying to do this via the DG papertape reader and papertape punch devices. Going from RDOS to linux was easy. Going from linux back to RDOS not so easy. HELP! I can't get the script to write out the binary file. It needs to be actual binary data not a text representation of binary data

The fix has to be simple. I just don't see it.

Notes: RDOS - Data General's Realtime Disk Operating System

RDOS -> linux convert CRs to LFs

linux -> RDOS convert LFs to CRs

Original RDOS Macro file output by the RDOS BPUNCH(binary punch) command.

Output to a simulated papertape punch. The papertape punch is simulated as an unstructured linux file.

MESSAGE INIT WORKING DIRECTORIES-SETUP.MC
MESSAGE
INIT MACROS
INIT UTIL
INIT FORT
DIR PETE
GDIR

[download]

Hexdump of the above macro file as a linux binary file, one byte for each character punched. Note the CR(0d) RDOS line endings.

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |.........
+.......|
*
00000190  4d 45 53 53 41 47 45 20  49 4e 49 54 20 57 4f 52  |MESSAGE I
+NIT WOR|
000001a0  4b 49 4e 47 20 44 49 52  45 43 54 4f 52 49 45 53  |KING DIRE
+CTORIES|
000001b0  2d 53 45 54 55 50 2e 4d  43 0d 4d 45 53 53 41 47  |-SETUP.MC
+.MESSAG|
000001c0  45 0d 49 4e 49 54 20 4d  41 43 52 4f 53 0d 49 4e  |E.INIT MA
+CROS.IN|
000001d0  49 54 20 55 54 49 4c 0d  49 4e 49 54 20 46 4f 52  |IT UTIL.I
+NIT FOR|
000001e0  54 0d 44 49 52 20 50 45  54 45 0d 47 44 49 52 0d  |T.DIR PET
+E.GDIR.|
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |.........
+.......|
*
00000380

[download]

ptreader.pl takes the above binary file, replaces the RDOS CR(od) line endings with linux LF(0a) line endings, strips off all of the nulls(00), and outputs a linux text file. Read() works fine to read the BPUNCH binary file.

#! /usr/bin/perl

# ptreader.pl - Papertape decoder for the Simh Data General Nova/Eclip
+se Simulator
#                   The linux file PTP.OUT is produced by the RDOS BPU
+NCH command
#                   PTP.OUT is a binary file that has a lot of leading
+ and trailing nulls
#                   The nulls are stripped off and the remainder is co
+pied to the output
#                   file PTP.TXT. Line endings are converted.
#
#                ptreader provides a way to copy text files from
#                   the simulated RDOS filesystem to the linux filesys
+tem
#
#                RDOS -> PTP.OUT -> ptreader.pl -> PTP.TXT
#
# Created by:    James M. Lynes jr.
# Created on:    January 29,2024
# Last Modified: 01/29/2024 - Initial version
#                01/30/2024 - Additional comments, fix line ending(0x0
+A)
#
# Usage:         On RDOS: BPUNCH filename
#                    Creates the (binary formatted) linux file: PTP.OU
+T
#
#                On linux: ./ptreader.pl
#                    Reads PTP.OUT and creates the reformated linux te
+xt file PTP.TXT
#
# Note:          Use ptwriter.pl to format a linux text file to be rea
+d by RDOS

use strict;
use warnings;

open(my $in, '<:raw', "PTP.OUT") or die;
open(my $out, '>', "PTP.TXT") or die;

my $content;
my $length;
my $maxlength = 4000;
my @inbytes;
my @outbytes;

$length = read($in, $content, $maxlength);       # Read the binary fil
+e
print "Length: $length\n";                       # Show total bytes re
+ad

@inbytes = unpack("C*", $content);               # Unpack bytes into a
+n array

foreach(@inbytes) {                              # Delete null bytes
    next if($_ == 0);
    push(@outbytes, $_);
}

foreach(@outbytes){printf ("0x%02X ", $_)};      # Show a hex dump of 
+array
print "\n";

foreach(@outbytes) {                              # Show the text read
+ 
    my $code = chr($_);
    
    if($_ == 0x0D) {
        $code = chr(0x0A);                        # Fix linux line end
+ing
    }

    print "$code";
    print $out $code;                            # Copy text to output
+ file
}

close $in;
close $out;

[download]

ptwriter is supposed to take a linux text file, change LFs back to CRs, and write out the result into a binary file. This code fails miserably! Actually getting it to write out the binary file is the problem.

#! /usr/bin/perl

# ptwriter.pl - Papertape encoder for the Simh Data General Nova/Eclip
+se Simulator
#                   The linux file PTR.IN is a binary formatted file w
+hich can 
#                   be read from the RDOS $PTR1 device.
#                   
#               The user is prompted for the linux text file to conver
+t
#
#               ptwriter provides a way to copy text files from
#                   the linux filesystem to the simulated RDOS filesys
+tem 
#
#               linux file -> ptwriter.pl -> PTR.IN -> RDOS 
#
# Created by:    James M. Lynes jr.
# Created on:    January 30,2024
# Last Modified: 01/30/2024 - Initial version
#                
#
#  Usage:        On linux: ./ptwriter.pl
#                    Creates the (binary formatted) linux file: PTR.IN
#                    User is prompted for the linux text file name
#         
#                On RDOS: XFER/A $PTR1 filename
#                    Copies linux file PTP.IN to RDOS file filename
#
#
# Note:          Use ptreader.pl to format a RDOS binary file into a l
+inux text file

use strict;
use warnings;

print "\n\n";
print "ptwriter - convert linux text file to RDOS papertape input file
+\n";
print "===============================================================
+\n";
print "Input linux text file name: ";

my $lfile = <STDIN>;
chomp($lfile);

open(my $in, '<', $lfile) or die;
open(my $out, '>', "PTR.IN") or die;

my @inbytes;
my @bytes;
my @outbytes;
my $outbytes;
my @binbytes;

while(my $line = <$in>) {                         # Read text file con
+vert to binary
    @bytes = unpack('C*', $line);
    push(@outbytes, @bytes);
}

my $len = $#outbytes + 1;
print "Characters Read: $len\n\n";

#foreach(@outbytes){printf ("0x%02X ", $_)};       # Show a hex dump o
+f array
#print "\n\n";

foreach(@outbytes) {
    my $code = $_;
    
    if($code == 0x0A) {                          # Convert linux line 
+end to RDOS line end
        $code = 0x0C;
    }
    
#    printf("0x%02X ", $code);                    # Show the converted
+ text
    push(@binbytes, $code);
}

#print "\n\n";
#foreach(@binbytes) {printf ("0x%02X ", $_)};     # *** Output data lo
+oks good at this point ***
#print "\n\n";


my $ctr = 0;
foreach(@binbytes) {                              # *** This code not 
+producing correct
    my $c = $binbytes[$ctr];                      #         binary fil
+e ***
    print "$c";
    print $out $c;
    $ctr = $ctr + 1;
}
    
close $in;
close $out;

[download]

James

There's never enough time to do it right, but always enough time to do it over...

Replies are listed 'Best First'.
Re: Why Is Writing to a Binary File so Hard?
by hv (Prior) on Jan 31, 2024 at 23:22 UTC

    You are converting characters to the integer value of their codepoints with the unpack(), and you end up printing those integers. At some point you need to convert integers back to characters with chr().

    The smallest change to fix this problem in your code would probably be to replace the print lines:

    #   print "$c";
    print chr($c);
#   print $out $c;
    print $out chr($c);

    [download]

    An alternative would be to make the whole thing much simpler by treating the line directly without converting to integers at all:

        while (my $line = <$in>) {
        $line =~ s/\n/\r/;  # replace LF with CR in the line
        print $out $line;   # and output it
    }

    [download]
[reply]
[d/l]
[select]
      Minor tweak to allow reading of a line containing "0" : 
         while (defined(my $line = <$in>)) {
        chomp $line;  # Zap trailing "\n" efficiently 
        print $out $line,"\r";
    }

      [download]

                      "If it happens once, it's a bug. If it happens twice, it's a feature. If it happens more than twice, it's a design philosophy."

[reply]
[d/l]

        Minor tweak to allow reading of a line containing "0" :

           while (defined(my $line = <$in>)) {
        chomp $line;  # Zap trailing "\n" efficiently 
        print $out $line,"\r";
    }

        [download]

        Not required:

        #!/usr/bin/perl

use strict;
use warnings;
use autodie;

open my $out,'>:raw',"file.tmp";
print $out "a line\na second line\n0";
close $out;

open my $in,'<:raw',"file.tmp";
while (my $line=<$in>) {
    chomp $line;
    print "read: $line\n";
}
close $in;

        [download]
        >perl foo.pl
read: a line
read: a second line
read: 0
>

        [download]

        Any line but the last in a file always contains a trailing newline, so $line is guaranteed to be true for all but the last line, and reading empty lines and lines containing only "0" is no problem for all but the last line:

        >perl -E '"0\n" and say "true"'
true
>perl -E '"\n" and say "true"'
true
>perl -E '"0" or say "false"'
false
> perl -E '"" or say "false"'
false
>perl -E 'undef or say "false"'
false
>

        [download]

        The last line WOULD BE a problem if perl would check for truth. But in the special case of readline a.k.a. <HANDLE>, it checks for definedness, not for truth. Quoting perlfunc:

        readline EXPR

        readline

        Reads from the filehandle whose typeglob is contained in EXPR (or from *ARGV if EXPR is not provided). In scalar context, each call reads and returns the next line until end-of-file is reached, whereupon the subsequent call returns undef. [...]

        This is the internal function implementing the <EXPR> operator, but you can use it directly.

        [...]If either a readline expression or an explicit assignment of a readline expression to a scalar is used as a while/for condition, then the condition actually tests for definedness of the expression's value, not for its regular truth value.

        (Emphasis mine)

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]
[select]

      The regex would eliminate one of the loops. Thanks for the reply.

      James

      There's never enough time to do it right, but always enough time to do it over...

[reply]
Re: Why Is Writing to a Binary File so Hard?
by NetWallah (Canon) on Jan 31, 2024 at 21:50 UTC
    Perhaps you need binmode on $out?

                    "If it happens once, it's a bug. If it happens twice, it's a feature. If it happens more than twice, it's a design philosophy."

[reply]

      The :raw does the same as binmode. Thanks for the reply.

      James

      There's never enough time to do it right, but always enough time to do it over...

[reply]

        The code you posted doesn't use :raw on your $out filehandle

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom