Re: pack() returns an unusable string
by choroba (Cardinal) on May 27, 2021 at 14:22 UTC
|
Your problem is unrelated to pack and can be easily demonstrated without it:
my $p = "X\x00Y";
system $^X, '-wE', "say '$p'";
The problem is the zero byte in an argument (at least on Linux).
map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
| [reply] [d/l] [select] |
Re: pack() returns an unusable string
by kcott (Archbishop) on May 27, 2021 at 08:45 UTC
|
G'day Rob,
I'm not sure if this helps, but it seems to reproduce what you're describing on a different platform: Cygwin 3.2.0 on Win10.
I kept most of your posted code as is; I added some info up front;
and, I looped through all of the templates on both 5.32 and 5.34.
$ cat pm_11133064_pack_test.pl
use warnings;
print "OS: $^O; Perl: $^V;\n",
`perl -V:usequadmath`, `perl -V:nvtype`, "\n";
my @templates = qw{d d< d> D D< D>};
for $template (@templates) {
print "TEMPLATE: $template\n";
$nv = 2.4;
$p = pack $template, $nv;
$s = "'$p'";
system $^X, '-wle', "print unpack('H*', $s);";
}
$ perl pm_11133064_pack_test.pl
OS: cygwin; Perl: v5.32.0;
usequadmath='undef';
nvtype='double';
TEMPLATE: d
3333333333330340
TEMPLATE: d<
3333333333330340
TEMPLATE: d>
4003333333333333
TEMPLATE: D
Invalid type 'D' in pack at pm_11133064_pack_test.pl line 11.
$ perl pm_11133064_pack_test.pl
OS: cygwin; Perl: v5.34.0;
usequadmath='undef';
nvtype='double';
TEMPLATE: d
3333333333330340
TEMPLATE: d<
3333333333330340
TEMPLATE: d>
4003333333333333
TEMPLATE: D
Can't find string terminator "'" anywhere before EOF at -e line 1.
TEMPLATE: D<
Can't find string terminator "'" anywhere before EOF at -e line 1.
TEMPLATE: D>
Can't find string terminator "'" anywhere before EOF at -e line 1.
| [reply] [d/l] [select] |
|
|
I'm not sure if this helps ...
Heh ... it helps ... but only in that it reaffirms what I already knew ;-)
I'll elaborate a little on how this all came about.
As I mentioned in the opening post, with perl-5.34.0, pack's 'D' templates are allowed on all builds irrespective of whether $Config{nvtype} is 'double' or 'long double' or '__float128'.
In earlier perl versions the 'D' templates were supported only if $Config{nvtype} was 'long double'.
With perl-5.34.0 and nvtype of 'long double' we get:
C:\>perl -wle "print unpack 'H*', pack 'D', 2.4;"
9a999999999999990040000000000000
whereas with perl-5.34.0 and nvtype of double, the same command yields a slightly different result:
C:\>perl -wle "print unpack 'H*', pack 'D', 2.4;"
00989999999999990040000000000000
That's pretty much as I expected because the string being unpacked in the first one-liner is different to the string being unpacked in the second one-liner.
But then I wanted to see what the result would be if the string created on the long double build (in the first one-liner) was fed to the 'double' build (in the second one-liner).
Would the result be 9a999999999999990040000000000000 or 00989999999999990040000000000000 ?
(I was quite sure it would be the former, but I knew that if I didn't check then I'd get it wrong.)
So I thought I'd do something cute:
I decided that, on my 'long double' build, I would run the following script:
use warnings;
$template = 'D<';
$nv = 2.4;
$p = pack $template, $nv;
$s = "'$p'";
# Now pass $p to the 'double' build of perl.
$double_perl = "C:/perl-5.34.0/bin/MSWin32-x64-multi-thread/perl.exe";
system $double_perl, '-wle', "print unpack('H*', $s);";
But, as already demonstrated, that doesn't DWIM.
In the end, I took the clunky approach of running 2 scripts.
Firstly, on the 'long double' build of perl-5.34.0 I ran the following script that would print the string returned by pack() to a file:
use warnings;
$p = pack("D", 2.4);
open WR, '>', 'packstr.txt' or die "Opening: $!";
binmode(WR);
print WR $p;
close WR or die "Closing: $!";
Then, on the 'double' build of perl-5.34.0, I ran a script that passed the string that was saved in packstr.txt to unpack():
use warnings;
open RD, '<', 'packstr.txt' or die "Opening: $!";
binmode(RD);
$p = <RD>;
close RD or die "Closing: $!";
print unpack("H*", $p);
That script output 9a999999999999990040000000000000.
(Hmph ... as if it was ever going to do anything else ...)
It just would have been so much cleaner and simpler if I could have got the first approach to work.
Cheers, Rob | [reply] [d/l] [select] |
Re: pack() returns an unusable string
by tybalt89 (Monsignor) on May 26, 2021 at 18:37 UTC
|
From your warning message, I would guess that your packed string '$p' has either a single quote "'" in it or an escape or $ or @ at the end
that eats the closing single quote.
Try looking at a dumped wersion of $p.
| [reply] |
|
|
I would guess that your packed string '$p' has either a single quote "'" in it or an escape or $ or @ at the end that eats the closing single quote
I should add that there's nothing special about the given NV value of 2.4.
The behaviour I'm seeing is happening for all NV values that I've tested.
To get a dump of $s:
# test.pl
use warnings;
use Devel::Peek;
$template = 'D<';
$nv = 2.4;
$p = pack $template, $nv;
$s = "'$p'";
Dump $s;
system $^X, '-wle', "print unpack('H*', $s);";
That outputs the following (which does, at least, contain an "@"):
C:\_32\pscrpt>perl test.pl
SV = PV(0x4bcf98) at 0x571248
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x4b7e88 "'\232\231\231\231\231\231\231\231\0@\0\0\0\0\0\0'"\0
CUR = 18
LEN = 20
Can't find string terminator "'" anywhere before EOF at -e line 1.
But if I change the value of $nv to 2.4e100, I get:
C:\_32\pscrpt>perl test.pl
SV = PV(0x3ecf98) at 0x383348
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x3e7e88 "'\vL\177v\257zb\206\264>\0\0\0\0\0\0'"\0
CUR = 18
LEN = 20
Can't find string terminator "'" anywhere before EOF at -e line 1.
Interestingly, changing the value to 3.1e-100 results in a different warning:
C:\_32\pscrpt>perl test.pl
SV = PV(0x26cf98) at 0x5311f8
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x267e88 "'9\302\271\243\"\211\224\255\264>\0\0\0\0\0\0'"\0
CUR = 18
LEN = 20
Use of uninitialized value $_ in print at -e line 1.
Perhaps something to do with the internal double quote.
For a value of 13.16e-100, I get:
C:\_32\pscrpt>perl test.pl
SV = PV(0x38cf98) at 0x4c51b8
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x387e88 "'\245^,\356\266\0208\270\266>\0\0\0\0\0\0'"\0
CUR = 18
LEN = 20
Can't find string terminator "'" anywhere before EOF at -e line 1.
Is there some way to pass the string returned by pack() to that system command ?
I assume that it ought to be possible, and that I'm just doing something dumb.
Cheers, Rob
| [reply] [d/l] [select] |
|
|
#!/usr/bin/perl -l
use strict; # https://perlmonks.org/?node_id=11133064
use warnings;
my $template = 'd<';
my $nv = 2.4;
$nv = 4135; # NOTE errors for this number
my $p = pack $template, $nv;
use Data::Dumper;
$Data::Dumper::Terse = $Data::Dumper::Useqq = 1;
my $s = Dumper $p;
print ">$s<\n";
system $^X, '-wle', "use strict; print unpack('H*', $s);";
Outputs:
>"\0\0\0\0\0'\260\@"
<
000000000027b040
note that it adds a newline, but perl doesn't care.
| [reply] [d/l] [select] |
|
|
|
|
|
|
|
#!/usr/bin/perl -l
use strict; # https://perlmonks.org/?node_id=11133064
use warnings;
my $template = 'd<';
my $nv = 2.4;
$nv = 4135; # NOTE errors for this number
my $p = pack $template, $nv;
use Data::Dump qw(pp);
my $s = pp $p;
print ">$s<\n";
system $^X, '-wle', "use strict; print unpack('H*', $s);";
Outputs:
>"\0\0\0\0\0'\xB0\@"<
000000000027b040
| [reply] [d/l] [select] |
|
|
# test.pl
use warnings;
$template = 'd<';
$nv = 2.4;
$p = pack $template, $nv;
$s = "'$p'";
print ">$s<\n";
system $^X, '-wle', "print unpack('H*', $s);";
prints:
>'333333@'<
3333333333330340
running a 64 bit build of 5.32. Note that there is an ETX character between the final 3 and the @ character, which, when you think about it, is the string that the unpack is showing us so for the success case there is a trailing @ and it isn't swallowing the trailing '.
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond
| [reply] [d/l] [select] |
|
|
OP said error only happens on D<, not d<, which doesn't run on my machine.
However, here is a number for which the error occurs on 5.32 with d<
#!/usr/bin/perl -l
use strict;
use warnings;
my $template = 'd<';
my $nv = 2.4;
$nv = 4135; # NOTE errors for this number
my $p = pack $template, $nv;
my $s = "'$p'";
print ">$s<\n";
system $^X, '-wle', "print unpack('H*', $s);";
Outputs:
>''°@'<
Can't find string terminator "'" anywhere before EOF at -e line 1.
which shows that my guess that the pack produces a single quote character is plausible.
| [reply] [d/l] [select] |
|
|
Re: pack() returns an unusable string
by ikegami (Patriarch) on May 29, 2021 at 03:08 UTC
|
Parameters can't contain the NUL character. Your so-called working code breaks if you change 2.4 to 2.5 (which generates at least one a NUL byte).
I recommend that you pass the hex of the packed string. You could also use base64, JSON, etc, etc, etc.
Furthermore, you have a code-injection bug. "'$p'" is not an appropriate way to generate a Perl string literal. $p could easily contain byte 0x27 (single quote). A trailing byte 0x5C (backslash) would also be problematic.
I recommend that you pass the value as an argument.
use strict;
use warnings;
my $template = 'd<';
my $nv = 2.5;
my $p = pack $template, $nv;
system $^X, '-wle', 'print $ARGV[0]', unpack('H*', $p);
Seeking work! You can reach me at ikegami@adaelis.com
| [reply] [d/l] [select] |
|
|
I recommend that you pass the value as an argument
What I was wanting to do was to pass the actual string as the argument.
It didn't really occur to me that doing that would be such a problem. I thought that people would be able to show me that this could be achieved with little fuss. (I was wrong ;-)
What I was interested in doing is easily demonstrated in the following script, where I've used the code that you supplied, but have replaced the last line with 2 lines:
use strict;
use warnings;
my $template = 'D<';
my $nv = 2.4;
my $p = pack $template, $nv;
# I'm running this script on perl-5.34.0
# with nvtype of 'long double'.
# I want to see the result of perl-5.34.0
# with nvtype of 'double' unpacking $p :
my $perl = "C:/perl-5.34.0/bin/MSWin32-x64-multi-thread/perl.exe";
system $perl, '-wle', 'print unpack("H*", $p)';
If that DWIMmed, it would output :
9a999999999999990040000000000000
but, of course, all that it emits is the warning:
Use of uninitialized value $_ in print at -e line 1.
Sure, I could take a leaf out of your book and alter my last line to:
system $perl, '-wle', 'print $ARGV[0]', unpack('H*', $p);
But that unpack('H*', $p) is being evaluated by the 'long double' build of perl.
I wanted to see what happened when it was evaluated by my 'double' build of perl-5.34.0.
I've since accomplished this by other means.
Cheers, Rob | [reply] [d/l] [select] |
|
|
| [reply] [d/l] [select] |
|
|
|
|
|
|
|