Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I have wrapped a dll called shmdll.dll that encapsulates some Shared Memory routines and am trying to access it with Win32::API. I know about the Win32MemMap module but it seems to have a small memory leak in even the simplest cases. Anyway, I have two C routines:

int __export __stdcall SetSharedMem(const int shareIndex, const void* +lpszBuf, const DWORD bufLength) int __export __stdcall GetSharedMem(const int shareIndex,void* lpszBuf +, const DWORD bufLength)
SetSharedMem writes what is passed in the variable lpszBuf (up to character bufLength) into a shared memory location referenced by shareIndex
GetSharedMem gets bufLength characters from that same shared memory location and puts it into lpszBuf

I have two corresponding Win32::API declarations:

$SetSharedMem = Win32::API->new('shmdll', 'int SetSharedMem(int shareI +ndex, VOID* lpszBuf, DWORD bufLength)'); $GetSharedMem = Win32::API->new('shmdll', 'int GetSharedMem(int shareI +ndex,VOID* lpszBuf, DWORD bufLength)');
I don't seem to be having a problem setting the shared memory location but when I get the shared memory location using:

$returnBuff = " " x ($str_length);<br> $return = $GetSharedMem->Call(0,$returnBuff,$str_length);<br><br>
it seems to stop before first null character. I dumped (using Devel::Peek Dump command) the buffer when the shared memory is being set and what was returned. That looks like the following:

Setting:
SV = PV(0x1abed40) at 0x1dea898<br> REFCNT = 1<br> FLAGS = (POK,pPOK)<br> PV = 0x1dda434 "\4\4\0041234\4\4\4\10\3\2\0\0\0\n\r20\0asdfasdfaf\3\0 +\0\0Red\4 \2\1\0\0\0\n\01710|\0|Asdfasdfaf\4\0\0\0Blue"\0<br> CUR = 69<br> LEN = 70<br><br>
Getting:
SV = PV(0x1abed64) at 0x1dea94c<br> REFCNT = 1<br> FLAGS = (POK,pPOK)<br> PV = 0x1dda218 "\4\4\0041234\4\4\4\10\3\2"\0<br> CUR = 13<br> LEN = 71<br><br>
As you can see by the PV value it cuts off before the three nulls in a row. I am using a memcpy in the GetSharedMemory routine like:

memcpy(lpszBuf,tempShmEntry.shmLpMapAddress,bufLength); {where tempShmEntry.shmLpMapAddress is the address of the base of the buffer}

I have written the information in lpszBuf to a file from the C dll and it is all there. There seems to be a problem between C an Perl however. Does anyone have any ideas?
Thanks in advance,
Mike

20040204 Edit by Corion Added code tags

Replies are listed 'Best First'.
Re: Problems Passing String Argument With Nulls From C To Perl
by Roger (Parson) on Feb 04, 2004 at 04:24 UTC
    Perl string can contain '\0', but '\0' marks the end of a string in C. There seems to be a problem somewhere in the string conversion from C back to Perl.

      Roger, That is true but I thought by passing a pointer value into the C procedure and using a memcpy (not a strcpy) with a string length I should be able to get around that. -Mike
        I have created the following test case to show how to preserve null (\0) characters during Perl/C string conversion. You need to use newSVpvn to convert C char * to Perl string while preserving '\0' in the char *. I suspect the Win32::API module uses newSVpvf or similar that does not preserve the null characters. I would roll my own ShareMem in XS instead of creating a separate DLL and accessing it via Win32::API.
        # ----- Makefile.PL ----- use ExtUtils::MakeMaker; WriteMakefile( 'NAME' => 'StringTest', 'VERSION_FROM' => 'StringTest.pm', 'PREREQ_PM' => {}, 'LIBS' => [], 'DEFINE' => '', 'INC' => '', ); # ----- StringTest.pm ----- package StringTest; require 5.00503; use strict; use warnings; require Exporter; require DynaLoader; our @ISA = qw(Exporter DynaLoader); our @EXPORT_OK = qw(set); our $VERSION = '1.00'; bootstrap StringTest $VERSION; 1; # ------ StringTest.xs ------ #include "EXTERN.h" #include "perl.h" #include "XSUB.h" MODULE = StringTest PACKAGE = StringTest SV *set(SV *s, IV l) CODE: { char *str = malloc(l); int i; memcpy(str, SvPVX(s, l), l); // OK // debug the copy printf("(l=%d) ", l); for (i=0;i<l;i++) { printf("%02x ", str[i]); } printf("\n"); RETVAL = newSVpvn(str, l); // newSVpvn preserves null free(str); } OUTPUT: RETVAL # ----- stest.pl ------ use strict; use StringTest qw/set/; my $str = "A\0B\0C\0"; # a string with nulls print "OLD: $str\n"; my $new_str = set($str, length($str)); # create a new string print "NEW: $new_str\n"; # ------ OUTPUT ------ OLD: A B C (l=6) 41 00 42 00 43 00 NEW: A B C

Re: Problems Passing String Argument With Nulls From C To Perl
by Anonymous Monk on Feb 04, 2004 at 04:34 UTC
    You obviusly have an issue with treating the data as a null terminated string somewhere in your code. What is the wrapper of which you speak? This looks a likely source of the issue if you can prove that a direct call to the DLL returns the expected stored data.
      The following are the procedures I use to set and get from the shared memory area. I threw the structure in as a reference. Ignore the red +s as they indicate a line wrap. -Mike
      //*********************************************************** //* * //* Structure Definitions * //* * //*********************************************************** struct shmInfoEntry { char shmBufferName[25]; DWORD shmBufferSize; DWORD shmDataSize; HANDLE shmHMapFile; LPVOID shmLpMapAddress; }; //*********************************************************** //*Name: SetSharedMem * //*Function: SetSharedMem sets the value of the shared * //* memory area reference by shareIndex * //*Return Vals: 0 = Operation Failed * //* 1 = Operation Succeeded * //*********************************************************** int __export __stdcall SetSharedMem(const int shareIndex, const void* +lpszBuf, const DWORD bufLength) { LPVOID recordOffset; struct shmInfoEntry tempShmEntry; // DEGUG FOR SEEING IF lpszBuf is writing out the proper stuff // FILE *crapfile; // // // Write out file // crapfile=fopen("crapfile.txt","wb"); // fwrite(lpszBuf,bufLength,1,crapfile); // fclose(crapfile); if (shareIndex > lastShmIndex) { // Have To Use A Existing Share return 0; } // Get Record Offset To Struct Of Info recordOffset = (DWORD) lpvMem + (sizeof(int) + (shareIndex * sizeO +fInfoStruct)); // Copy Struct From Shared Memory memcpy(&tempShmEntry,recordOffset,sizeOfInfoStruct); if (bufLength > tempShmEntry.shmBufferSize) { //NOTE: THIS IS BAD, RESIZE BUFFER OR FAIL return 0; } // Set Data Size tempShmEntry.shmDataSize = bufLength; // Save Data Size memcpy(recordOffset,&tempShmEntry,sizeOfInfoStruct); // lpMapAddress should be the first memory byte of the shared memo +ry struct memcpy(tempShmEntry.shmLpMapAddress,lpszBuf,bufLength); return 1; } //*********************************************************** //*Name: GetSharedMem * //*Function: GetSharedMem gets the value of the shared * //* memory area reference by shareIndex * //*Return Vals: 0 = Operation Failed * //* 1 = Operation Succeeded * //*********************************************************** int __export __stdcall GetSharedMem(const int shareIndex,void* lpszBuf +, const DWORD bufLength) { LPVOID recordOffset; struct shmInfoEntry tempShmEntry; // FILE *crapfile; if (shareIndex > lastShmIndex) { return 0; } // Get Record Offset To Struct Of Info recordOffset = (DWORD) lpvMem + (sizeof(int) + (shareIndex * sizeO +fInfoStruct)); // Copy Struct From Shared Memory memcpy(&tempShmEntry,recordOffset,sizeOfInfoStruct); if (bufLength > tempShmEntry.shmBufferSize) { //NOTE: THIS IS BAD, RESIZE BUFFER OR FAIL return 0; } // Grab Stuff Out Of Buffer memcpy(lpszBuf,tempShmEntry.shmLpMapAddress,bufLength); // Debugging what should be returned // crapfile=fopen("crapfile.txt","wb"); // fwrite(lpszBuf,bufLength,1,crapfile); // fclose(crapfile); return 1; }
        I have updated the reply to your post ... here 326410

Re: Problems Passing String Argument With Nulls From C To Perl
by lamberms (Sexton) on Feb 04, 2004 at 19:18 UTC
    Everyone, I figured out a hokey way to make this all work with the Win32::API module. I changed the Win32::API::Type module as follows:

    1. I changed the Unpack subroutine as follows
    sub Unpack { my $type = $_[0]; # ADDED if ($type eq "MYBUFFER*") { DEBUG "(PM)Type::Unpack: got packing 'c', is a pointer, unpack +ing 'Z*' '$_[1]'\n"; $_[1] = unpack("A*", $_[1]); DEBUG "(PM)Type::Unpack: returning '$_[1]'\n"; return $_[1]; } elsif(packing($type) eq 'c' and is_pointer($type)) { DEBUG "(PM)Type::Unpack: got packing 'c', is a pointer, unpack +ing 'Z*' '$_[1]'\n"; $_[1] = unpack("Z*", $_[1]); DEBUG "(PM)Type::Unpack: returning '$_[1]'\n"; return $_[1]; } DEBUG "(PM)Type::Unpack: unpacking '".packing($type)."' '$_[1]'\n" +; $_[1] = unpack( packing($type), $_[1]); DEBUG "(PM)Type::Unpack: returning '$_[1]'\n"; return $_[1]; }

    2. I added the following to the TYPE section

    MYBUFFER                c

    3. I added the following to the POINTER section

    LPBUFFER                MYBUFFER

    Then I changed my procedure declaration to GetShareMem as follows:

    $GetSharedMem = Win32::API->new('shmdll', 'int GetSharedMem(int shareI +ndex,MYBUFFER* lpszBuf, DWORD bufLength)');


    Now I get the correct thing in the lpszBuf (Whoopee). Thanks for all your help!
    Best Regards
    Mike