hotpelmen has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

Could you please kindly help with printing empty line into report when using Perl v5.32?

Background: while transferring an old Perl application (a collection of scripts) from RHEL7 (Perl 5.16) to RHEL9 (Perl 5.32), found some tests failing in 5.32 because whitespace value refuses to get printed on its own line into report when format uses picture line that starts with caret and ends with two tildes.

Removing tildes (or running the following sample code in Perl 5.16) results in whitespace printed on its own line without issues.

I wrote the following script for the purpose of illustration of the phenomenon. Note: same behavior is displayed when using "format NAME = ..." syntax instead of "$format = ..." + "eval $format". 

#!/usr/bin/perl

use strict;
use warnings;

my ($date, $reference, $rep_line);

my $format = q/format REP_TOP =
@<<<<<<<<<<<<<<< @>>>>>>>>>>>>>>
$date, $reference
.
/;
eval $format;

$format = q/format REP =
^<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ~~
$rep_line
.
/;
eval $format;

$date = "11/11/2011";
$reference = "REF123";

open my $fh, ">", "/tmp/test.txt" or die "$!\n";

$fh->format_name("REP");
$fh->format_top_name("REP_TOP");
$fh->autoflush(1);
$fh->format_lines_per_page(10);

for (@ARGV) {
    $rep_line = $_;
    write $fh;
}

[download]
Running this script as follows prints only lines with non-whitespace values: 
$ perl testformat.pl 0 1 " " 2 3 " " 4 5;
$ cat /tmp/test.txt
11/11/2011                REF123
0
1
2
3
4
5

[download]
Expected (and as it works with 5.16): 
$ perl testformat.pl 0 1 " " 2 3 " " 4 5;
$ cat /tmp/test.txt
11/11/2011                REF123
0
1

2
3

4
5

[download]
Assuming I still want to use format, how would you advise to overcome this and achieve printing of whitespace values into separate lines of report? I need to keep ~~ there to be able to wrap longer values. It appears that ~~ is confused by Perl with single ~. Any idea WHY the behavior changed? Is it a bug?

Replies are listed 'Best First'.
Re: perl 5.32 format question
by haukex (Archbishop) on May 29, 2023 at 12:15 UTC

    I reduced your code down to this test case: perl -e '$_=" "; formline q{^<<<~~}, $_; length $^A or die' and a bisect reported that "9b4bdfd4 is the first bad commit". This is a commit that fixes several bugs in formats; it was first released as part of Perl v5.20.0 9 years ago.

    If you read the documentation, note that a double tilde means "the line will be repeated until all the fields on the line are exhausted, i.e. undefined", however, a single tilde "anywhere in the line" means to suppress "lines where all fields are blank". Although I think the documentation could be a lot more clear on this (Update: patch applied), a double tilde implies the single tilde, as is confirmed by the relevant code in Perl 5.0.0 and today.

    Anyway, the following provides your expected output both on older and recent Perl versions (tested on 5.14 to 5.36):

    format REP =
^<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
$rep_line
^<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ~~
$rep_line
.

    [download]
[reply]
[d/l]
[select]
      Thank you very much, haukex for your prompt and informative response as well as for submitting the patch to enhance the documentation. The format you suggested worked beautifully.
[reply]
Re: perl 5.32 format question
by kcott (Archbishop) on May 29, 2023 at 13:17 UTC

    G'day hotpelmen,

    The oldest version I have is v5.30.0, so I couldn't attempt to replicate your results. I had a look in perlform for a number of versions but couldn't find any change. My look was brief; you may want to study more closely:

    I suspect you may be right about '~' & '~~'. There does seem to be some conflict between "perlform: Suppressing Lines Where All Fields Are Void" and "perlform: Repeating Format Lines".

    "Assuming I still want to use format, how would you advise to overcome this and achieve printing of whitespace values into separate lines of report?"

    This code, pm_11152452_testformat.pl, worked for me. 

    #!/usr/bin/env perl

use strict;
use warnings;
use autodie;
use constant REP_LINE_MAX_LENGTH => 3;

# For demo only
print "Perl version: $^V\n";

my $outfile = '/tmp/pm_11152452_test.txt';

my ($date, $reference, $rep_line);

format REP_TOP =
@<<<<<<<<<<<<<<< @>>>>>>>>>>>>>>
$date, $reference
.

format REP =
^*~~
$rep_line
.

$date = '11/11/2011';
$reference = 'REF123';

open my $fh, '>', $outfile;

$fh->format_name('REP');
$fh->format_top_name('REP_TOP');
$fh->autoflush(1);
$fh->format_lines_per_page(10);

for (@ARGV) {
    $rep_line = substr($_, 0, REP_LINE_MAX_LENGTH);
    write $fh;
}

# For demo only
system cat => $outfile;
unlink $outfile;

    [download]

    I kept REP_LINE_MAX_LENGTH very short for demo purposes; you'll want it to be 33 (if I counted correctly). I tested it on three versions: 

    $ ./pm_11152452_testformat.pl zero one " " two three " " four five
Perl version: v5.30.0
11/11/2011                REF123
zer
one

two
thr

fou
fiv

$ ./pm_11152452_testformat.pl zero one " " two three " " four five
Perl version: v5.32.0
11/11/2011                REF123
zer
one

two
thr

fou
fiv

$ ./pm_11152452_testformat.pl zero one " " two three " " four five
Perl version: v5.36.0
11/11/2011                REF123
zer
one

two
thr

fou
fiv

    [download]

    See how you go with 5.16.

    — Ken

[reply]
[d/l]
[select]
      Thank you kcott, your help is much appreciated! I guess ^*~~ would be practical if one wants to do their own wrapping (or trimming, like you did) of the printed values. It's interesting that in the format you suggested, tildes do not have the same effect on code's ability to create blank lines. One can wonder if it's intentional or an omission.
[reply]

Back to Seekers of Perl Wisdom