in reply to Re^2: Storable- long integer size
in thread Storable- long integer size

linux-x64 $ perl -V:'.*size|byte.*' | sort
byteorder='12345678';
charsize='1';
d_chsize='undef';
d_malloc_good_size='undef';
d_malloc_size='undef';
doublesize='8';
fpossize='16';
gidsize='4';
i16size='2';
i32size='4';
i64size='8';
i8size='1';
intsize='4';
ivsize='8';
longdblsize='16';
longlongsize='8';
longsize='8';
lseeksize='8';
nvsize='16';
ptrsize='8';
shortsize='2';
sig_size='69';
sizesize='8';
st_ino_size='8';
u16size='2';
u32size='4';
u64size='8';
u8size='1';
uidsize='4';
uvsize='8';
$

[download]
hp-ux ia64 $ perl -V:'.*size|byte.*' | sort
byteorder='87654321';
charsize='1';
d_chsize='undef';
d_malloc_good_size='undef';
d_malloc_size='undef';
doublesize='8';
fpossize='8';
gidsize='4';
i16size='2';
i32size='4';
i64size='8';
i8size='1';
intsize='4';
ivsize='8';
longdblsize='16';
longlongsize='8';
longsize='8';
lseeksize='8';
nvsize='16';
ptrsize='8';
shortsize='2';
sig_size='49';
sizesize='8';
u16size='2';
u32size='4';
u64size='8';
u8size='1';
uidsize='4';
uvsize='8';
$

[download]

The quotation will be different on Windows. An IV has not only length shown by ivsize, but also an order, as shown by byteorder. Both have to match for Storable to work. I also would promote the use of Sereal instead, as that was written with portability in mind.

You can also keep using Storable if what you store are strings from pack where you define the format yourself. A long would then be packed using "l>", and unpacked with the same signature: the ">" defines the endianness


Enjoy, Have FUN! H.Merijn

Replies are listed 'Best First'.
Re^4: Storable- long integer size
by syphilis (Archbishop) on Oct 21, 2015 at 13:36 UTC
    An IV has not only length shown by ivsize, but also an order, as shown by byteorder. Both have to match for Storable to work

    Yes, I knew that ivsize and byteorder would have to match for Storable to work, but I was unsure whether ivtype also had to match.

    I take it that ivtype does *not* have to match.
    Thanks !!

    Cheers,
    Rob
[reply]
      I take it that ivtype does *not* have to match.

      That may be so, but it seems that longsize also has to match - because that's apparently what Storable stipulates(and for no good reason, AFAICT).
      On an Ubuntu box using a perl whose ivsize was 8 and byteorder was 12345678, I ran: 
      use strict;
use warnings;
use Storable;

my $val = 12345678;
my $file = 'store.ubu';

store (\$val, $file);

my $valref = retrieve($file);
print "$val $$valref\n";

      [download]
      Then I copied (via scp) the generated 'store.ubu' over to a Windows box. (The file does not get changed by the transfer.) Using a compatible Windows perl (ivsize of 8 and byteorder of 12345678) I ran this script to retrieve the value: 
      use strict;
use warnings;
use Storable;

my $file = 'store.ubu';

print "Retrieving $file\n";

my $r = retrieve($file);
print $$r, "\n";

      [download]
      It output: 
      Retrieving store.ubu
Long integer size is not compatible at C:/_64/perl522_492/lib/Storable
+.pm line 383, at retrieve.pl line 9.

      [download]
      I then went into the Storable-2.51 source and changed line 6037 of Storable.xs from: 
      if ((int) *current++ != sizeof(long))
to
if ((int) *current++ != sizeof(IV))

      [download]
      (That's the condition that terminates the script and emits the "Long integer size is not compatible" error message.)
      I then rebuilt and installed this modified Storable-2.51 and re-ran the retrieval script. This time I got exactly what (I think) I should have been getting all along: 
      Retrieving store.ubu
12345678

      [download]
      Storable is actively disabling retrieval when it's not necessary to do that.
      Is that the way it's supposed to behave ?
      Why is it effectively insisting that longsize be the same - especially given that longsize typically differs between linux and windows ?

      Cheers,
      Rob
[reply]
[d/l]
[select]
        Why is it effectively insisting that longsize be the same - especially given that longsize typically differs between linux and windows ?

        My guess is that its because no one who appreciates the difference between LP64 & LLP64 programming models has really looked at the code since it was ported to 64-bit.

        The code simply makes no attempt at differentiation. The magic_check function from Storable.xs :

        /*
 * magic_check
 *
 * Make sure the stored data we're trying to retrieve has been produce
+d
 * on an ILP compatible system with the same byteorder. It croaks out 
+in
 * case an error is detected. [ILP = integer-long-pointer sizes]
 * Returns null if error is detected, &PL_sv_undef otherwise.
 *
 * Note that there's no byte ordering info emitted when network order 
+was
 * used at store time.
 */
static SV *magic_check(pTHX_ stcxt_t *cxt)
{

    ...


    current = buf + c;
    
    /* sizeof(int) */
    if ((int) *current++ != sizeof(int))
        CROAK(("Integer size is not compatible"));

    /* sizeof(long) */
    if ((int) *current++ != sizeof(long))
        CROAK(("Long integer size is not compatible"));

    /* sizeof(char *) */
    if ((int) *current != sizeof(char *))
        CROAK(("Pointer size is not compatible"));

    if (use_NV_size) {
        /* sizeof(NV) */
        if ((int) *++current != sizeof(NV))
            CROAK(("Double size is not compatible"));
    }

    return &PL_sv_undef;    /* OK */
}

        [download]

        (The comment could be read to suggest another, less generous, interpretation.)

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

In Section Seekers of Perl Wisdom