in reply to Re: History repeats: 128 bit Math:Bigint numbers in a 64 bit DBI world
in thread History repeats: 128 bit Math:Bigint numbers in a 64 bit DBI world

Math:Int128 didn't work out for me under Strawberry PERL.

The build tests failed during compilation.

Anyway I came up with the following.

use strict; use Benchmark::Timer; use Math::BigInt; our $twoto32=Math::BigInt->bone(); $twoto32 <<= 32; our $twoto64=Math::BigInt->bone(); $twoto64 <<= 64; my $t=Benchmark::Timer->new(); $t->start('overall'); for (my $j=10000;$j<100000;$j++) { my $test_address='0x'.'ABCD' x 6 . '123'.$j; #my $test_address='0x'.'4142' x 6 . '30313233'; # check for bit and by +te order => ascii "ABABABABABAB0123" $t->start('new_bigint'); my $bigint=Math::BigInt->new($test_address); $t->stop('new_bigint'); $t->start('convert_to'); my $binary16=bigint_to_varbinary16( $bigint); #print "binary16".$binary16." length ".length($binary16)."\n"; $t->stop('convert_to'); $t->start('convert_from'); my $bigint2= varbinary16_to_bigint($binary16); $t->stop('convert_from'); $t->start('convert_to_64bit'); my $binary16=bigint_to_varbinary16_64bit( $bigint); #print "binary16".$binary16." length ".length($binary16)."\n"; $t->stop('convert_to_64bit'); $t->start('convert_from_64bit'); my $bigint2= varbinary16_to_bigint_64bit($binary16); $t->stop('convert_from_64bit'); #print $bigint." ".$bigint2."\n"; } $t->stop('overall'); print $t->reports; exit 0; sub bigint_to_varbinary16 { my $bigint = shift; my $my_copy=$bigint->copy(); # create a new bigint that can get mung +ed my @oa; #output array my $i=4; while($i>1) { $i--; #my ($q,$r) = $my_copy->brsft(32); # shift off 4 octets on the rig +ht. In my install $r is NOT set! my ($q,$r) = $my_copy->bdiv($twoto32); # shift off 4 octets on the + right $oa[$i]= pack 'N',$r; # 32 bits in network bit order (MSB first) if ($i==1) { $oa[0]= pack 'N',$q; # saves one bdiv last; } } return join('',@oa); } sub varbinary16_to_bigint { my $binary16=shift; my $result = Math::BigInt->new(0); my $i=0; my $p=0; while ($i<4) { $result <<= 32 if $i>0; #!4294967296; $result += unpack ('N', substr $binary16, $p , 4); $p+=4; $i++; } ; return $result; } sub bigint_to_varbinary16_64bit { my $bigint = shift; my $my_copy=$bigint->copy(); # create a new bigint that can get mung +ed my ($q,$r) = $my_copy->bdiv($twoto64); # shift off 8 octets on the r +ight. brsft does not set remainder on my test machine, hence bdiv return reverse join('', pack('Q',$r), pack('Q',$q)); # Q is little e +ndian on my machine. I want DB SELECT to also be able to perform comp +arison operators. } sub varbinary16_to_bigint_64bit { my $binary16=shift; my $string = reverse $binary16; my $result = Math::BigInt->bzero(); $result += unpack ('Q', substr $string, 8 , 8); $result <<= 64; $result += unpack ('Q', substr $string, 0 , 8); return $result; }

Unsurprisingly, 64 bit pack and unpack is more efficient. I just wish there was an equivalent built in BigInt function to do this, and this was also built into DBI. But I ain't got time/skills to do that so I'll have to roll my own.

results:

new_bigint90000 trials of new_bigint (4.193s total), 46us/trial convert_to90000 trials of convert_to (17.848s total), 198us/trial convert_from90000 trials of convert_from (31.125s total), 345us/trial convert_to_64bit90000 trials of convert_to_64bit (7.648s total), 84us/ +trial convert_from_64bit90000 trials of convert_from_64bit (13.012s total),1 +44us/trial

Replies are listed 'Best First'.
Re^3: History repeats: 128 bit Math:Bigint numbers in a 64 bit DBI world
by salva (Canon) on Jul 04, 2016 at 08:40 UTC
    Math:Int128 didn't work out for me under Strawberry PERL.
    Math::Int128 requires a compiler supporting 128bit integers. In practice that means it should work under the 64bit version of Strawberry Perl but not under the 32bit one.
Re^3: History repeats: 128 bit Math:Bigint numbers in a 64 bit DBI world
by syphilis (Archbishop) on Jul 04, 2016 at 08:44 UTC
    Math:Int128 didn't work out for me under Strawberry PERL

    It's incompatible with 32-bit Strawberry Perl but should be trivial to build and install on 64-bit Strawberry Perl.

    Cheers,
    Rob
Re^3: History repeats: 128 bit Math:Bigint numbers in a 64 bit DBI world
by BrowserUk (Patriarch) on Jul 04, 2016 at 10:07 UTC

    As already identified, get yourself a 64-bit perl.

    I cannot conceive of any reason to use a 32-bit perl (or most anything else) under a 64-bit OS; it's like buying a V12 Ferrari and disconnecting the plugs to one bank of the V.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice. Not understood.
      You guys have given the "correct" answer: I'm also convinced that Math::Int128 is the right way to go.

      Math::Int128 *should* work on my install, but it does not.

      perl -v This is perl 5, version 16, subversion 3 (v5.16.3) built for MSWin32-x +64-multi-thread cpan Math::Int128 <snip> t/00-report-prereqs.t ..... ok t/author-eol.t ............ skipped: these tests are for testing by th +e author t/author-no-tabs.t ........ skipped: these tests are for testing by th +e author t/author-pod-spell.t ...... skipped: these tests are for testing by th +e author t/bugs.t .................. ok t/Math-Int128.t ........... 1/? # Failed test 'max int128 >> 95' # at t/Math-Int128.t line 174. # got: '4294967295' # expected: '1' # $max >> $j = 4294967295, $max / 2 ** $j = 1, $max = 1701411834604692 +3173168730 3715884105727, $j = 95, 2 ** $j = 39614081257132169000000000000.000000 +, int128(2 ** $j) = 39614081257132168796771975168 # Failed test 'max int128 >>= 95' # at t/Math-Int128.t line 183. # got: '4294967295' # expected: '1' t/Math-Int128.t ........... Dubious, test returned 148 (wstat 37888, 0 +x9400) Failed 2/785 subtests t/Math-UInt128.t .......... 1/? # Failed test 'max uint128 >> 95' # at t/Math-UInt128.t line 172. # got: '8589934591' # expected: '1' # Failed test 'max uint128 >>= 95' # at t/Math-UInt128.t line 176. # got: '8589934591' # expected: '1' <snip>
      Obviously there are math overflows being triggered in the tests. I'll try another (later) version but there's some very old code on this box. Plus I need mod_perl2 and Apache2 which increases the dependencies. Not asking for sympathy. It's just how it is.

      BTW pack ('Q', blah) and unpack ('Q',blah) which require 64 bit PERL, DOES work.

        If you do 'make clean' and then 'makefile'; do you see this?:

        ... Checking for unsigned int __attribute__ ((__mode__ (DI)))... no Checking for uint64_t... (cached) no Checking for unsigned int __attribute__ ((__mode__ (DI)))... (cached) +no It looks like your compiler doesn't support a 128-bit integer type ( +one of "int __attribute__ ((__mode__ (TI)))" or "__int128"). One of these t +ypes is necessary to compile the Math::Int128 module.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice. Not understood.
        There's a known bug with Strawberry 5.16.x builds - see line 179 of Math-Int128.t.
        I suspect you might have uncovered another bug with Strawberry 5.16.3.

        For my own MinGW builds of MSWin32-x64-multi-thread I find no problem with Math-Int128-0.22 going back to 5.12.0.

        Cheers,
        Rob