in reply to Sending an array through a socket!

A network packet is a string of bytes and so to put something into a packet, you first have to turn it into a string of bytes. There are lots of ways to do that, but you want to do it in a way where the other side can unambiguously turn the string of bytes that it receives back into something resembling the original data structure.

This is called marshaling.

You second example implies that the elements of your arrays are all strings of length 8 composed of "0"s and "1"s. Is this really true?

If instead of that you had an array of numbers, then you could use: send( SOCKET, pack("C*",@array), 0, $paddr ) to send those elements over the socket and then use @array= unpack( "C*", $packet ); on the other side to pull the elements back out.

However, you say "all of this data I need to send in binary (not ascii)". But there are lots of quite different meanings to the term "binary":

  1. The (text) representation of a number using base 2
  2. A string of bytes that contains bytes that aren't "just plain text"
  3. The compact native format for data that the computer uses
  4. A type of tree where each node can have two children
  5. A method for searching a sorted list by cutting the candidates in half at each step
Since we are dealing with data over a socket and because you mention "ASCII" (which is often used to mean "just plain text"), I'd think you'd mean (2) or (3). But your second example implies that you have data that is already been converted to (1). If someone told me to use "binary and not ASCII" for sending some numbers over a socket, I'd assume they meant (3) [which also implies (2)]. But I don't understand why you'd have those number in format (1) beforehand.

But if that is really what you have, then there are lots of ways to build your packet. Here are two examples:

$packet= pack "b*", join "", @array; $packet= join "", map { pack "b8", $_ } @array;
But I'd really prefer to have more information to convince me that this is really what you should be doing. (:

        - tye (but my friends call me "Tye")

Replies are listed 'Best First'.
Re: (tye)Re: Sending an array through a socket!
by flipxor (Initiate) on Oct 05, 2001 at 06:24 UTC
    I guess I was a bit vague. Thanks for the help, I learned a lot from your example
    What I want to do is interpret a bunch of values from a web
    page put these values in an array and convert them into a home grown
    packet and send this through a socket. And I want them to be a string of 1s and 0s.
    What I'm having a hard time with Perl (being a newbie) is
    that I'm used to declaring unsigned 8 bit ints, 16 bit ints, etc.
    The format I want to send as a packet are as so:
    B: An 8 bit signed integer
    Mgmt: 8 bit signed integer
    ClassV: 8 bit signed integer
    R: this is one bit long
    Method: this is 7 bits long (int)
    Status: 16 bits signed integer
    ClassSpec: 16 signed int
    TransID: 64 bit signed int
    Rez: 16 bits signed int
    AttMod: 32 bit signed int
    and the Data: is 1856 bits long, can be anything.
    So if I did a tcp dump, the way I want the packets to look like are:

    0x06 0x01 0x07 0x00
    0x00 0x00 0x00 0x00
    0x00 0x00 0x10 0x00
    0x61 0x1e 0x00 0x00
    0x02 0x00 0x00 0x00
    I've used the pack functions, but what I'm having difficulty is
    with the one bit long R value and the 7 bit long Meth int.
    and I have to throw it out in 1s and 0s. Thanks....

      Ah, that makes more sense. But first, let's address one point where I think you are still confused:

      And I want them to be a string of 1s and 0s.

      Well, we are talking computers here so everything can be thought of as a sequence of 1s and 0s. But that is something completely different from a string of "1"s and "0"s, which is what you were previously creating.

      The string "110001" is (on computers using ASCII) a string of bytes having values 49, 49, 48, 48, 48, and 49, in that order. Which you could also think of as the sequence of bits 00110001 00110001 00110000 00110000 00110000 00110001. See my point?

      BTW, you could also think of that as the sequence of bits 10001100 10001100 00001100 00001100 00001100 10001100. You see, the order of bits within a byte is a matter of convention and not everyone uses the same convention. Luckilly, this doesn't matter because the only things that talk to each other 1 bit at a time are designed by people who are very careful to have both sides agree on the convention. So this "problem" turns out to hardly ever be a problem for software guys.

      See, the byte has become the (well, nearly) universal unit of data exchange. So don't worry about the bits. Worry about the bytes.

      This also means that if you want to send a 16-bit integer, you have two choices of convention: big endian and little endian. Unfortunately, this problem you do have to worry about.

      One convention is that "network" data should use big-endian values. So you might want to go with that and stop worrying about it (unless you have to talk to someone who might not have gone with that, *sigh*).

      Anyway, on to the answer... Your packet is very easy to construct:

      $packet= pack( "c c c C n n NN n N", $B, $Mgmt, $ClassV, ($R<<7)|$Method, $Status, $ClassSpec, $TransID_Hi, $TransID_Lo, $Rez, $AttMod ) . $Data;
      Getting the data out on the other end is a bit more complex because unpack doesn't signed versions of "n" and "N" (we don't care when using pack since it "just deals").

      Anyway, the part you were having problems with is best solved by not dealing with "1"s and "0"s but with 1s and 0s. That is, shift bits around rather than convert numbers to strings.

      Sorry, I'm out of time. I hope that explanation helped some. (:

              - tye (but my friends call me "Tye")
        Understood. I had one question about your packet construction, this part:
        $ClassSpec, $TransID_Hi, $TransID_Lo, $Rez, $AttMod )
        . $Data;


        confuses me.... Why is there a . before $data and $data is not part of the pack?
        I'm apologizing before hand, but this is my call for help because I just can't write code very well anymore
        Or am just not thinking.... probably the latter ;).
        I'm still not getting it to work for me. My newbiness has definitely gotten me in a bind

        Maybe this is my massive bit of desperation. But I can implement what I want to in C (when creating the packet)
        But I still need Perl to interpret the data from a web page.
        Could I just possibly interface this C code with my perl to generate the packet
        for me? Perl will just pass the values through to my C code with the correct arguments?
        #include "Simple.h" #include "Mad.h" #include "RcvIpcPorts.h" #define MAD_SIZE 512 void longswapheader(MadHeader *pmh) { U32 i; U32 *pl = (U32 *)pmh; for ( i = 0; i < sizeof(MadHeader) ; i+=4, pl++ ) { *pl = ntohl(*pl); } } int SendMad(U8 *buffer, int sd, struct sockaddr_in *send_addr, U16 att +r) { MadHeader *pmh; fd_set fds; int bytes; printf("Press Enter to Send...\n"); getchar(); pmh = (MadHeader *)buffer; pmh->BaseVersion = 0; pmh->ClassId = Mad_Class_V_ComMgt; pmh->ClassVersion = 1; pmh->R = 0; pmh->MethodId = Method_CM_Send; /* 3== send */ pmh->Status = 0; pmh->ClassSpecific = 0; pmh->TransactionId = 0; pmh->AttributeId = attr; pmh->rsv0 = 0; pmh->AttributeModifier = 0; longswapheader(pmh); FD_ZERO(&fds); FD_SET(sd, &fds); if ((select(32/*sd+1*/, (fd_set *) 0, &fds, (fd_set *) 0, (struct +timeval *) 0) < 1) || !FD_ISSET(sd, &fds)) { printf("select failed err %d\n", errno); return -1; } else { bytes = sendto(sd, buffer, MAD_SIZE, 0, (__SOCKADDR_ARG)send_a +ddr, sizeof(struct sockaddr_in)); } return bytes; } int main(int argc, char *argv[]) { int sd, rc; int passive = 0; struct sockaddr_in addr, recv_addr, send_addr; MadHeader *pmh; U8 send_buffer[MAD_SIZE], receive_buffer[MAD_SIZE]; CommID lcid, rcid; sd= socket(PF_INET, SOCK_DGRAM, 0); printf("socket sd %d\n", sd); if ( sd == -1 ) { printf("socket failed\n"); } memset(&addr, 0, sizeof(addr)); memset(&recv_addr, 0, sizeof(recv_addr)); memset(&send_addr, 0, sizeof(send_addr)); memset(send_buffer, 0, sizeof(send_buffer)); memset(receive_buffer, 0, sizeof(receive_buffer)); addr.sin_family = PF_INET; addr.sin_addr.s_addr = INADDR_ANY; send_addr.sin_family = AF_INET; if ( (argc > 1) && (strcmp(argv[1],"passive") == 0) ) { passive = 1; send_addr.sin_port = htons(CMA_P_PORT); addr.sin_port = htons(IBIS_P_PORT); } else { passive = 0; send_addr.sin_port = htons(CMA_A_PORT); addr.sin_port = htons(IBIS_A_PORT); } send_addr.sin_addr.s_addr = inet_addr("127.0.0.1"); if ( bind(sd, (struct sockaddr *)&addr, sizeof(addr)) != 0 ) { printf("bind failed\n"); } pmh = (MadHeader *)send_buffer; int i; if ( passive ) { for ( i=0; i < 1; i=1) { // receive a REQ rc = ReceiveMad(receive_buffer, sd, &recv_addr); if ( rc < 0 ) continue; lcid = ((MadPacket *)receive_buffer)->Payload.REQ.commID; rcid = ((MadPacket *)receive_buffer)->Payload.REQ.rsv0; printf("lcid %lu rcid %lu\n", lcid, rcid); // send REP ((MadPacket *)send_buffer)->Payload.REP.commID = lcid; ((MadPacket *)send_buffer)->Payload.REP.remCommID = rcid; rc = SendMad(send_buffer, sd, &send_addr, Attr_CM_ConnectR +eply); if ( rc < 0 ) continue; // receive an RTU rc = ReceiveMad(receive_buffer, sd, &recv_addr); if ( rc < 0 ) continue; // receive DREQ rc = ReceiveMad(receive_buffer, sd, &recv_addr); if ( rc < 0 ) continue; // send DREP ((MadPacket *)send_buffer)->Payload.REP.commID = lcid; ((MadPacket *)send_buffer)->Payload.REP.remCommID = rcid; rc = SendMad(send_buffer, sd, &send_addr, Attr_CM_Disconne +ctReply); if ( rc < 0 ) continue; } } else { for ( i=0; i < 1; i=1) { // send REQ rc = SendMad(send_buffer, sd, &send_addr, Attr_CM_ConnectR +equest); if ( rc < 0 ) continue; // receive REP rc = ReceiveMad(receive_buffer, sd, &recv_addr); if ( rc < 0 ) continue; lcid = ((MadPacket *)receive_buffer)->Payload.REP.commID; rcid = ((MadPacket *)receive_buffer)->Payload.REP.remCommI +D; printf("lcid %lu rcid %lu\n", lcid, rcid); ((MadPacket *)send_buffer)->Payload.RTU.commID = lcid; ((MadPacket *)send_buffer)->Payload.RTU.remCommID = rcid; // send RTU rc = SendMad(send_buffer, sd, &send_addr, Attr_CM_ReadyToU +se); if ( rc < 0 ) continue; // send DREQ ((MadPacket *)send_buffer)->Payload.DREQ.commID = lcid; ((MadPacket *)send_buffer)->Payload.DREQ.remCommID = rcid; rc = SendMad(send_buffer, sd, &send_addr, Attr_CM_Disconne +ctRequest); if ( rc < 0 ) continue; // receive DREP rc = ReceiveMad(receive_buffer, sd, &recv_addr); if ( rc < 0 ) continue; } } close(sd); return 0; }
        So that's my C. But my ugly arse PERL code looks like this:
        #madtest.cgi - perl script that interprets a user inputted mad for #agent testing. use Socket; use Sys::Hostname; %postInputs = readPostInput(); my ( $count, $hisiaddr, $hispaddr, $histime, $host, $iaddr, $paddr, $port, $proto, $rin, $rout, $rtime, $SECS_of_70_YEARS, $setipaddr); $setipaddr = $postInputs{'IPAddress'}; $ipaddr = inet_aton($setipaddr); $PORTNO = 7777; $MAD_SIZE = 512; $proto = getprotobyname('udp'); $paddr = sockaddr_in($PORTNO, $ipaddr); # port, ipaddress @madArray = ($postInputs{'BaseVersion'}, $postInputs{'MgmtClass'} +, $postInputs{'ClassVersion'}, $postInputs{'Method'}, (pack "s", $pos +tInputs{'Status'}), (pack "s", $postInputs{'ClassSpecific'}), (pack +"s", $postInputs{'AttributeID'}), $postInputs{'Reserved'}, (pack "l", + $postInputs{'AttributeModifier'}), (pack "Z",$postInputs{'Data'})); $packet= pack( "c c c C n n NN n N a*", $postInputs{'BaseVersion'}, $postInputs{'MgmtClass'}, $postInpu +ts{'ClassVersion'}, ($postInputs{'R'}<<7)|$postInputs{'Method'}, $pos +tInputs{'Status'}, $postInputs{'ClassSpecific'}, $postInputs{TransactionID_Hi}, $T +ransID_Lo, $postInputs{'Reserved'}, $postInputs{'AttributeModifier'} ,$postInputs{'Data'}); #Next Throw data out to a UDP Server (Our Interprocess Communication.) # #Go into binary mode binmode (SOCKET); select(socket(SOCKET, PF_INET, SOCK_DGRAM, $proto)) || die "socket: +$!"; #bind(SOCKET, $paddr) || die "bind: $!"; #$| = 1; $z2=0; while ($z2 < $madCount) { send(SOCKET, $packet, 0, $paddr); #this doesn't work, obviously. :( send(SOCKET, $packet, MAD_SIZE, $pad +dr); $z2++; } close(SOCKET) || die "dead socket: $!"; select(STDOUT); printThankYou(); ####Method that splits up pair/values sub readPostInput(){ my (%searchField, $buffer, $pair, @pairs); if ($ENV{ 'REQUEST_METHOD'} eq 'POST'){ read(STDIN, $buffer, $ENV{ 'CONTENT_LENGTH'}); #separate the name values into an array named @pairs @pairs = split (/&/, $buffer); #Array is then Url decoded into $searchField hash. foreach $pair (@pairs){ ($name, $value) = split(/=/, $pair); $value =~ tr/+/ /; $value =~ s/%([a-fA-F0-9][a-fA-F0-9]) /pack("C", hex($1))/eg; $name =~ tr/+/ /; $name =~ s/%([a-fA-F0-9][a-fA-F0-9]) /pack("C", hex($1))/eg; $searchField{ $name} = $value; } } return (%searchField); } #return a web page sub printThankYou(){ # no need to actually put this in for example. }

        2001-10-13 Edit by Corion : Added CODE tags