perl 5.32 format question

hotpelmen has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

Could you please kindly help with printing empty line into report when using Perl v5.32?

Background: while transferring an old Perl application (a collection of scripts) from RHEL7 (Perl 5.16) to RHEL9 (Perl 5.32), found some tests failing in 5.32 because whitespace value refuses to get printed on its own line into report when format uses picture line that starts with caret and ends with two tildes.

Removing tildes (or running the following sample code in Perl 5.16) results in whitespace printed on its own line without issues.

I wrote the following script for the purpose of illustration of the phenomenon. Note: same behavior is displayed when using "format NAME = ..." syntax instead of "$format = ..." + "eval $format".

#!/usr/bin/perl

use strict;
use warnings;

my ($date, $reference, $rep_line);

my $format = q/format REP_TOP =
@<<<<<<<<<<<<<<< @>>>>>>>>>>>>>>
$date, $reference
.
/;
eval $format;

$format = q/format REP =
^<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ~~
$rep_line
.
/;
eval $format;

$date = "11/11/2011";
$reference = "REF123";

open my $fh, ">", "/tmp/test.txt" or die "$!\n";

$fh->format_name("REP");
$fh->format_top_name("REP_TOP");
$fh->autoflush(1);
$fh->format_lines_per_page(10);

for (@ARGV) {
    $rep_line = $_;
    write $fh;
}
[download]

Running this script as follows prints only lines with non-whitespace values:

$ perl testformat.pl 0 1 " " 2 3 " " 4 5;
$ cat /tmp/test.txt
11/11/2011                REF123
0
1
2
3
4
5
[download]

Expected (and as it works with 5.16):

$ perl testformat.pl 0 1 " " 2 3 " " 4 5;
$ cat /tmp/test.txt
11/11/2011                REF123
0
1

2
3

4
5
[download]

Assuming I still want to use format, how would you advise to overcome this and achieve printing of whitespace values into separate lines of report? I need to keep ~~ there to be able to wrap longer values. It appears that ~~ is confused by Perl with single ~. Any idea WHY the behavior changed? Is it a bug?

Comment on perl 5.32 format question Select or Download Code

Replies are listed 'Best First'.
Re: perl 5.32 format question by haukex (Archbishop) on May 29, 2023 at 12:15 UTC
I reduced your code down to this test case: `perl -e '$_=" "; formline q{^<<<~~}, $_; length $^A or die'` and a bisect reported that "`9b4bdfd4` is the first bad commit". This is a commit that fixes several bugs in `format`s; it was first released as part of Perl v5.20.0 9 years ago. If you read the documentation, note that a double tilde means "the line will be repeated until all the fields on the line are exhausted, i.e. undefined", however, a single tilde "anywhere in the line" means to suppress "lines where all fields are blank". Although I think the documentation could be a lot more clear on this (Update: patch applied), a double tilde implies the single tilde, as is confirmed by the relevant code in Perl 5.0.0 and today. Anyway, the following provides your expected output both on older and recent Perl versions (tested on 5.14 to 5.36): `format REP = ^<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< $rep_line ^<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ~~ $rep_line .` [download]	[reply] [d/l] [select]
Re^2: perl 5.32 format question by hotpelmen (Scribe) on May 30, 2023 at 15:59 UTC
Thank you very much, haukex for your prompt and informative response as well as for submitting the patch to enhance the documentation. The format you suggested worked beautifully.	[reply]
Re: perl 5.32 format question by kcott (Archbishop) on May 29, 2023 at 13:17 UTC
G'day hotpelmen, The oldest version I have is `v5.30.0`, so I couldn't attempt to replicate your results. I had a look in `perlform` for a number of versions but couldn't find any change. My look was brief; you may want to study more closely: https://perldoc.perl.org/5.16.0/perlform https://perldoc.perl.org/5.32.0/perlform https://perldoc.perl.org/5.36.1/perlform I suspect you may be right about '`~`' & '`~~`'. There does seem to be some conflict between "perlform: Suppressing Lines Where All Fields Are Void" and "perlform: Repeating Format Lines". "Assuming I still want to use format, how would you advise to overcome this and achieve printing of whitespace values into separate lines of report?" This code, `pm_11152452_testformat.pl`, worked for me. #!/usr/bin/env perl use strict; use warnings; use autodie; use constant REP_LINE_MAX_LENGTH => 3; # For demo only print "Perl version: $^V\n"; my $outfile = '/tmp/pm_11152452_test.txt'; my ($date, $reference, $rep_line); format REP_TOP = @<<<<<<<<<<<<<<< @>>>>>>>>>>>>>> $date, $reference . format REP = ^*~~ $rep_line . $date = '11/11/2011'; $reference = 'REF123'; open my $fh, '>', $outfile; $fh->format_name('REP'); $fh->format_top_name('REP_TOP'); $fh->autoflush(1); $fh->format_lines_per_page(10); for (@ARGV) { $rep_line = substr($_, 0, REP_LINE_MAX_LENGTH); write $fh; } # For demo only system cat => $outfile; unlink $outfile; [download] I kept `REP_LINE_MAX_LENGTH` very short for demo purposes; you'll want it to be `33` (if I counted correctly). I tested it on three versions: `$ ./pm_11152452_testformat.pl zero one " " two three " " four five Perl version: v5.30.0 11/11/2011 REF123 zer one two thr fou fiv $ ./pm_11152452_testformat.pl zero one " " two three " " four five Perl version: v5.32.0 11/11/2011 REF123 zer one two thr fou fiv $ ./pm_11152452_testformat.pl zero one " " two three " " four five Perl version: v5.36.0 11/11/2011 REF123 zer one two thr fou fiv` [download] See how you go with `5.16`. — Ken	[reply] [d/l] [select]
Re^2: perl 5.32 format question by hotpelmen (Scribe) on May 30, 2023 at 16:23 UTC
Thank you kcott, your help is much appreciated! I guess ^*~~ would be practical if one wants to do their own wrapping (or trimming, like you did) of the printed values. It's interesting that in the format you suggested, tildes do not have the same effect on code's ability to create blank lines. One can wonder if it's intentional or an omission.	[reply]


laziness, impatience, and hubris
	PerlMonks