ateague has asked for the wisdom of the Perl Monks concerning the following question:

UPDATE:
This issue was caused by my ignorance of my Perl's pointer size (8 bytes in this case). Packing the pointer into an integer of the correct size "fixes" the issue I was experiencing. Thank you tye and ikegami for your help!

(It is obvious now in hindsight, but maybe the Perldocs for pack/unpack could be updated to mention the 'p/P' templates and Perl's pointer size?)




Good evening!

I am trying to more clearly learn how pack/unpack works with values returned from Win32::API, specifically pointers and how to dereference them. I have a small example program that calls a "cat" function in a "test.dll" that takes two strings and returns them concatenated together.

Setting the output parameter in the function template to char* works as expected: 

#!/usr/bin/perl
use 5.018;
use strict;
use warnings;
use Win32::API;
use Data::Dumper;

$Win32::API::DEBUG = 1;

my $method = new Win32::API::More('test.dll', 'char* cat(char* a, char
+* b)');
if( !defined $method ) {
    die "Can't import API [cat]: $^E\n";
}

my $return = $method->Call('my ', 'potato');

print Dumper $return; # prints 'my potato'

[download]

Setting the output parameter to int in the function template returns a valid pointer (I can peek at that memory address and see the concatenated string), and I can dereference it with various combinations of pack/unpack, but I do not know why some work: 

#!/usr/bin/perl
use 5.018;
use strict;
use warnings;
use Win32::API;
use Data::Dumper;

$Win32::API::DEBUG = 1;

my $method = new Win32::API::More('test.dll', 'int cat(char* a, char* 
+b)');
if( !defined $method ) {
    die "Can't import API [cat]: $^E\n";
}

my $return = $method->Call('my ', 'potato');

print Dumper $return; # pointer address: 42952784

# works as expected: 'my potato'
my $x = unpack 'p', pack 'J', $return;
print Dumper $x;

# works as expected: 'my potato'
my $x = unpack 'p', pack 'Q', $return;
print Dumper $x;

# does not work as expected: undef
my $x = unpack 'p', pack 'I', $return;
print Dumper $x;

# does not work as expected: undef
my $x = unpack 'p', pack 'L', $return;
print Dumper $x;

# does not work as expected: undef
my $x = unpack 'p', pack 'N', $return;
print Dumper $x;

# Update: I made a typo in my original post.
# This (obviously) will not work on little-endian machines
#my $x = unpack 'p', pack 'N2', $return;
#print Dumper $x;

# works(??): 'my potato'
#my $x = unpack 'p', pack 'V2', $return;
#print Dumper $x;

# works(??): 'my potato'
my $x = unpack 'p', pack 'IN', $return;
print Dumper $x;

[download]

What exactly is going on with the different combinations of pack/unpack?

Perl info: 

perl -v

This is perl 5, version 18, subversion 4 (v5.18.4) built for MSWin32-x
+64-multi-thread

[download]

USE_64_BIT_INT has been enabled as well.
Version 0.82 of Win32-API

Thank you for your time

Replies are listed 'Best First'.
Re: Dereferencing a pointer returned from Win32::API with unpack 'p' (size)
by tye (Sage) on Aug 10, 2015 at 21:19 UTC

    You just need to pack an integer of the right size. Looks like 1) your build of Perl uses 8-byte pointers, 2) your system is little-endian, and 3) the pointer value that you are getting fits into a 4-byte integer. So you get success when you either pack an 8-byte integer or you pack a 4-byte (little-endian) integer followed by a 4-byte zero.

    The only helpful thing I can think of is that you can determine how big of a pointer your Perl wants with something like length( pack "P", "q" ).

    (Updated.)

    - tye        

[reply]
[d/l]
Re: Dereferencing a pointer returned from Win32::API with unpack 'p'
by ikegami (Patriarch) on Aug 10, 2015 at 21:58 UTC

    The size of a pointer varies between different builds of Perl (even on the same system). The size of a pointer (in bytes) is available from 

    perl -V:ptrsize

    [download]
    and 
    perl -E"use Config qw( %Config ); say $Config{ptrsize};"

    [download]

    There is no pack format that's guaranteed to always produces exactly the size of a pointer on all systems. That means you need to determine the correct format from the aforementioned variable if you want a portable solution: 

    use Config qw( %Config );

my $ptr_size = $Config{ptrsize};
my $ptr_format =
   $ptr_size == 8 ? 'Q' :
   $ptr_size == 4 ? 'L' :
   die("Unrecognized pointer size");

my $s = unpack 'p', pack $ptr_format, $addr;

    [download]

    You have a pointer size of 8 bytes, so "Q" is the appropriate format for you.

    To my knowledge, Windows only runs on little-endian systems. On a little-endian system, if you use an integer type that's too large, you'll simply add trailing zeros that will be ignored by unpack 'p'. 

    >perl -wE"say unpack 'p', pack('P', 'abc')"
abc

>perl -wE"say unpack 'p', pack('P', 'abc').qq{\0\0\0\0\0\0\0\0}"
abc

    [download]

    As such, you can safely use "J" on such a system since the IV and UV types are guaranteed to be large enough to hold a pointer. 

    my $s = unpack 'p', pack 'J', $addr;

    [download]

    Some types should be avoided since they have unpredictable size: "I".

    Some types should be avoided since they don't necessarily use native byte order: "N", "n", "V", "v". 

    # works(??): 'my potato'
my $x = unpack 'p', pack 'N2', $return;
print Dumper $x;

    [download]

    huh??? Aren't you on an x86 machine? That's a little-endian machine. On a little-endian system such as yours, "J" and "N2" will give different result, so they can't possibly both work. 

    >perl -wE"say sprintf '%vX', pack 'J', 0x0000000012345678"
78.56.34.12.0.0.0.0

    [download]
    >perl -wE"say sprintf '%vX', pack 'N2', 0x0000000012345678"
12.34.56.78.0.0.0.0

    [download]

    # works(??): 'my potato'
my $x = unpack 'p', pack 'IN', $return;
print Dumper $x;

    [download]

    You got lucky there. 

    >perl -wE"say sprintf '%vX', pack 'J', 42952784"
50.68.8F.2.0.0.0.0

>perl -wE"say sprintf '%vX', pack 'IN', 42952784"
50.68.8F.2.0.0.0.0

    [download]

    But it would have failed for larger addresses since your "I" only produces 4 bytes. 

    >perl -wE"say sprintf '%vX', pack 'J', 0x0123456789ABCDEF"
EF.CD.AB.89.67.45.23.1

>perl -wE"say sprintf '%vX', pack 'IN', 0x0123456789ABCDEF"
EF.CD.AB.89.0.0.0.0

    [download]

    When packing, there's no difference between corresponding signed and unsigned integer formats. I only mentioned unsigned pack formats above, but everything I said applies to the signed formats too.

[reply]
[d/l]
[select]

      Thank you for your explanations.

      huh??? Aren't you on an x86 machine? That's a little-endian machine. On a little-endian system such as yours, "J" and "N2" will give different result, so they can't possibly both work.
      You are correct. I made a mistake when copying/pasting my tests. It should be "V2". I have updated the original question.
[reply]
A reply falls below the community's threshold of quality. You may see it by logging in.

Back to Seekers of Perl Wisdom