jepri has asked for the wisdom of the Perl Monks concerning the following question:

For my IPC module I am sending things through a 'stream' that might have to be chopped into bits and then reassembled at the other end. I originally tried to mark out individual messages by something hard to occur by accident, such as "MESSAGE_DELIMITER" and use some cunning regexps... didn't work.

Should I go to 9-bit codewords, or use the modem - style coding ( I think it's V.90bis or something similar). Or maybe send a header which tells how long the rest of the message is (true packet-style).

More importantly... anyone got a module to do it for me?

____________________
Jeremy
I didn't believe in evil until I dated it.

Replies are listed 'Best First'.
Re: Stream delimiting chars
by grinder (Bishop) on Sep 13, 2001 at 20:38 UTC
    • If you are working with fixed length data, you can indicate the beginning with a specific character, e.g. 0xc0, and then you know you must assumble $length{0xc0} bytes.
    • If you are not dealing with truly binary data (e.g. all 8 bit values are possible), you can take a character that is not part of the alphabet used, and then bracket the data with that, e.g. 0xf0 . $foo_data . 0xf0
    • If you are dealing with truly binary data, and/or variable length data, the only true way to do it right is encode the length at the begin of the packet

    Not telling you much you don't know, eh? I guess you can take this as being a confirmation of what you already suspect.

    Be that as it may, as far as modules are concerned, this is dreadfully easy to put together with a bit of encoding and decoding via pack/unpack.

    --
    g r i n d e r
[reply]
      Yep. Variable length binary data. Looking at pack it seems uuencoding is the easiest. Shame it's not the most efficient.

      Thanks.

      ____________________
      Jeremy
      I didn't believe in evil until I dated it.

[reply]

        Just use "\n" as your end-of-record character (so you can use <SOCK>, even though it is often a bad idea) and do s/([\n\\])/\\$1/g before sending and s/\\./"n"eq$1?"\n":$1/ge; after you receive it.

                - tye (but my friends call me "Tye")
[reply]
[d/l]
[select]
Re: Stream delimiting chars
by jettero (Monsignor) on Sep 13, 2001 at 20:32 UTC
    I'm not quite sure... but this feels like what you want. 
    NAME
       IPC::Msg - SysV Msg IPC object class

SYNOPSIS
           use IPC::SysV qw(IPC_PRIVATE S_IRWXU);
           use IPC::Msg;

           $msg = new IPC::Msg(IPC_PRIVATE, S_IRWXU);

           $msg->snd(pack("l! a*",$msgtype,$msg));

           $msg->rcv($buf,256);

           $ds = $msg->stat;

           $msg->remove;

    [download]
[reply]
[d/l]
      I'm actually using IPC::SysV to do the transport (some of the time). IPC::SysV can only move a certain amount of data per message (quite small, about 4k).

      I've wrapped IPC::SysV in a module that sends data through and rebuilds it... thus the need for spotting the end of the message.

      I was being coy about the transport because I'm also using TCP, and anything else that comes to hand...

      ____________________
      Jeremy
      I didn't believe in evil until I dated it.

[reply]
Re: Stream delimiting chars
by suaveant (Parson) on Sep 13, 2001 at 20:50 UTC
    It is not a bad idea at all to have a fixed length header, which specifies the length of the message, read the 20 byte header or whatever, and read until you get the message length specified by the header...

                    - Ant
                    - Some of my best work - Fish Dinner

[reply]

Back to Seekers of Perl Wisdom