bliako has asked for the wisdom of the Perl Monks concerning the following question:

Wise Monks,

I am trying to translate some JS code into Perl. The problem is that I think said JS code does not handle well overflows when doing a left shift and it calculates the result (let's say a checksum) wrongly. But this is now the result and I must emulate that result in my Perl translation. I can not fix the JS code, it is not mine. Question is: how?

Here is the gist of it assuming a 64bit Linux console with node.js installed and bash:

node -e 'console.log(1<<30);' 1073741824 echo $((1<<30)) 1073741824 # same result perl -e 'print 1<<30, "\n"' 1073741824 # same result node -e 'console.log(1<<31);' -2147483648 # overflow in 32bit! echo $((1<<31)) 2147483648 # correct result perl -e 'print 1<<31, "\n"' 2147483648 # same result # to obtain the overflow in bash echo $((1<<63)) -9223372036854775808 # overflow in 64bit # still perl does not perl -e 'print 1<<63, "\n"' 9223372036854775808 perl -e 'print 1<<64, "\n"' 0 # gotcha! # this is when I realised the problem: node -e 'console.log(1169367104<<5);' -1234958336 echo $((1169367104<<5)) 37419747328 perl -e 'print 1169367104<<5, "\n"' 37419747328 # same result

(note: if you don't have node.js open your browser, open its 'web developer tools', go to tab 'console' and just paste that bit of harmless js code

So, I am asking: how can I make perl print -1234958336 when left-shifting 1169367104<<5 ?

use Test::More; is(1169367104<<5, -1234958336, "simulated JS 32bit shift."); done_testing;

bw, bliako

Edit: I have asked a more general question about JS/C/Perl integer overflows at https://stackoverflow.com/questions/77584450/emulate-javascripts-32bit-signed-integer-arithmetic-to-c-or-perl-some-discr

Replies are listed 'Best First'.
Re: Trying to translate overflowing JS code to Perl
by Corion (Patriarch) on Nov 28, 2023 at 19:21 UTC

    It seems that the Javascript is doing 32-bit signed math. So the easiest way is to do your calculations and then pack it into 32bit as 4 bytes, and then to reinterpret these four bytes as signed integer:

    perl -E 'say unpack "l", pack "L", (1169367104<<5)' -1234958336

      Pack Man Wow! thanks Corion that does it sub _leftshift32 { my ($n, $s) = @_; unpack "l", pack "L", ($n << $s) }

      I probably now have to implement all other arithemtic operators used in that JS code as above.

        I probably now have to implement all other arithmetic operators used in that JS code as above.

        I can't come up with a better solution ... but if you were to use a perl whose ivsize was 4, then I think (not rigorously tested) you could take care of it with a simple use integer; at the top of the script.
        On perl-5.38.0, built with just that configuration:
        D:\>perl -V:ivsize ivsize='4'; D:\>perl -le "print 1169367104<<5;" 3060008960 D:\>perl -Minteger -le "print 1169367104<<5;" -1234958336
        Cheers,
        Rob

        I believe you can do the complete calculation without conversion and just at the end downgrade to 32 bit and convert to signed. The following should work:

        sub calc_32bit_signed( $value ) { return unpack( "l", pack "L", ($value & 0xffffffff))) } ... say calc_32bit_signed( $n << 6 ); say calc_32bit_signed( $n + 2 );
        package Math32 { use Inline 'C', <<~'END'; #include <stdint.h> long calc(long input) { return (int32_t)((int32_t)input << 5); } END } use v5.36; say Math32::calc(1169367104);
        -1234958336

        Then all you need to do is write all the JS code in C, using int32_t variables and typecasts. If you want to declare the function parameter and return type as int32_t, you'd need to add it to the typemap, which is a hassle.

Re: Trying to translate overflowing JS code to Perl
by cavac (Prior) on Dec 01, 2023 at 12:51 UTC

    I don't know exactly what your JS code does. But there are many more design/concept differences between Perl and JS, besides overflow handling.

    Yes, you can translate JS code into Perl, but depending on what your end goal is, you might not have to. You can always integrate JS into your Perl script with something like JavaScript::Embedded. This uses the Duktape Javascript engine.

    Here's a basic "Helper" directly from my own framework from file JavaScript.pm:

    package PageCamel::Helpers::JavaScript; #---AUTOPRAGMASTART--- use v5.38; use strict; use diagnostics; use mro 'c3'; use English; use Carp qw[carp croak confess cluck longmess shortmess]; our $VERSION = 4.3; use autodie qw( close ); use Array::Contains; use utf8; use Data::Dumper; use Data::Printer; use builtin qw[true false is_bool]; no warnings qw(experimental::builtin); use PageCamel::Helpers::UTF; #---AUTOPRAGMAEND--- use JavaScript::Embedded; use JSON::XS; sub new($class, %config) { my $self = bless \%config, $class; if(!defined($self->{reph})) { croak('PageCamel::Helpers::JavaScript needs reph reporting han +dler'); } if(!defined($self->{timeout})) { croak('PageCamel::Helpers::JavaScript needs timeout (default t +imeout value)'); } my $js = JavaScript::Embedded->new(timeout => $self->{timeout}); $self->{js} = $js; $self->{js}->set('debuglog' => sub { $self->_logfromjs($_[0]); }); $self->{js}->eval(qq{ var memory = new Object; function __encode(obj) { return JSON.stringify(obj); } function __decode(txt) { return JSON.parse(txt); } function __setmemory(txt) { memory = __decode(txt); } function __getmemory() { return __encode(memory); } function __getKeys(obj) { var keys = Object.keys(obj); return keys; } }); # if(defined($self->{code})) { # $self->load(); # } return $self; } sub _logfromjs($self, $text) { $self->{reph}->debuglog($text); return; } sub loadCode($self, $code) { if(!defined($code)) { croak("JS code undefined!"); } $self->{code} = $code; $self->{js}->eval($self->{code}); return; } sub call($self, $name, @arguments) { my $func = $self->{js}->get_object($name); if(!defined($func)) { print STDERR "Function $func does not exist!\n"; return; } return $func->(@arguments); } sub registerCallback($self, $name, $func) { $self->{js}->set($name, $func); return; } sub encode($self, $data) { return encode_json $data; } sub decode($self, $json) { return decode_json $json; } sub toArray($self, $object) { my @arr; $object->forEach(sub { my ($value, $index, $ar) = @_; push @arr, $value; }); return @arr; } sub getKeys($self, $object) { my $rval = $self->call('__getKeys', $object); return $self->toArray($rval); } sub toHash($self, $object) { my @keys = $self->getKeys($object); my %hash; foreach my $key (@keys) { $hash{$key} = $object->$key; } return %hash; } sub setMemory($self, $memory) { $self->call('__setmemory', $memory); return; } sub getMemory($self) { return $self->call('__getmemory'); } sub initMemory($self) { $self->call('initMemory'); return; } 1;

    My stuff does a bit more than strictly required for your project, i assume. My module also implements the basics for simulating a memory-persistant object (which higher levels of my implementation dynamically load/store in a PostgreSQL database).

    PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP

      Thanks for this demonstration, I remember you mentioned this a few weeks ago perhaps. JS code calculates a weird checksum based on integers, length of string and string content. Nothing fancy

Re: Trying to translate overflowing JS code to Perl
by bliako (Abbot) on Nov 30, 2023 at 20:12 UTC

    Based on Corion's unpack( "l", pack "L", ($value & 0xffffffff))) I have put together a package which overloads basic arithmetic operators in an attempt to do these operations with 32bit integers thus simulating JS's 32bit integers. It succeeds in 1169367104<<5 = -1234958336 but fails for 1073741824+1073741824 (that's 1<<30 + 1<<30) (Perl result overflows: -2147483648 but it does not in JS: 2147483648). I have also tried NERDVANA's method of converting to 32bit in C and also with doing the addition above in C. These fail as well for said test case (1073741824+1073741824)

    5 mins edit - sorry: Well 1073741824 = 1000000000000000000000000000000 = 1<<30 (that's 31 digits leaving the 32nd for sign), so 1073741824+1073741824=1000000000000000000000000000000+1000000000000000000000000000000 should overflow into the sign bit. Producing negative zero (so to speak). But Perl with 32bit integers (and Corion's hack) produces -2147483648 (ok perhaps i am missing the details on what happens with the value-part in overflowing situations) and (node.js) JS produces 2147483648 as if no overflow occured. Perhaps I have got JS's default integer size wrong? Nope! results are for node.js!!! when I enter 1<<30 + 1<<30 into firefox JS console, I get 0. Which is the negative zero. Why Perl produces -2147483648. Which takes the single bit which is ON to be both value and sign?

    2hour Edit: I have changed the package code slightly to use Corion's method by default.

    Here is Math::AnyInt:

    And here is a basic test harness:

    bw, bliako

      And here is a basic test harness:

      I think your implementation of NERDVANA's code is flawed.
      In specifying a return type of "long", the returned "int32_t" is being cast to a 64-bit integer on most 64-bit systems.
      Instead of "long", you should specify a return type of "int32_t" - though I think you'll get away with specifying "int" (thereby avoiding the typemap issue that NERDVANA mentioned), which is the same as "int32_t" everywhere.
      (By "everywhere", I mean "everywhere that I've come across", though I believe the standards actually allow that "int" can be the same as "int64_t".)

      With that change in place, you should see the last 2 tests fail, as they ought (in their present form).

      Thanks for the fun - and for going to the trouble of presenting Math::AnyInt.

      Cheers,
      Rob
        Instead of "long", you should specify a return type of "int32_t" - tho +ugh I think you'll get away with specifying "int" (thereby avoiding t +he typemap issue that NERDVANA mentioned), which is the same as "int3 +2_t" everywhere.

        Thanks for the suggestion (and in the other threads too!). But when I specify a int32_t as return type to NERDVANA's function via Inline::C it compiles fine but Perl complains that it can not find that function. I think it checks the signatures and does not consider a function with return as int32_t as a proper candidate. It works fine for int (and long) return type.

Re: Trying to translate overflowing JS code to Perl
by Corion (Patriarch) on Jan 11, 2024 at 15:24 UTC

    There now is Math::JS, which implements the JS math. I currently see some test failures for 0.01 which seem to relate to floating point differences and some other failures. So this is likely not yet ready for prime time but one to watch.