I posted this to comp.lang.perl.misc and have gotten nada thus far. Maybe some monkly advice can shine down on me...

What I need to do: send data over a network in network-byte order (big endian) in 4 byte chunks (XDR data format description, RFC 1832). Integers are 4 bytes long, strings of variable size are preceded by an int (4 bytes) which gives the real data length and then null padded to be a multiple of 4 bytes long (sorta pascal-esque), etc. A byte is defined as 8 bits.

Two problems (at least) I've found thinking through this:

  1. First I don't know how to use pack/unpack in a way that I can convert perl data into big-endian, binary forms of the correct sizes (I assume I need to use the "N" template for pack/unpack).
  2. I'd like to start working on string data as soon as I possibly can; I don't want to wait for the complete string to be in the buffer before I can pass it along to needy clients. Let's say I've encoded "Hello, world!" to an XDR string where the size is not known ahead of time. It should be 20 bytes long (first 4 bytes is an int containing the size of the string, 13, the next 16 bytes will be "Hello, world!\0\0\0"). Now say I have the first 8 bytes of this datagram, I decode the first 4 bytes and learn the string is 13 chars long, now what I would like to do is decode the next 4 bytes and get that much of the string ("Hell"). How can I pack the original perl string in a way that these 4 byte boundaries can be observed when unpacking?

This is perl-to-perl over the net for now but will need to be perl-to-C very soon.

Also wondering, can length, substr, join, // and other various perl string functions be used to look at/modify packed data/bytes safely? Any pitfalls here?

Lastly is there documentation on pack/unpack anywhere that describes in more detail than the Camel book (specificly I'd love to see a lot a lot of commented examples). pack/unpack has always been a time sink for me when I've needed to use it.

thanks,
Micah

In reply to How to use pack/unpack data over a network by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.