EHLO,

I'm thinking a bit lately about multiplexing, it started with the only problem I can see for FTP protocol in modern internet - the need for additional tcp connections. ssh and scp solve this by creating virtual connections inside single TCP/IP channel. Basicaly ssh provides powerfull generic infrastructure for multiplexing lots of channels into one encrypted channel.

Another very visible place you can find such solutions is multimedia - container files with audio and video plexed together...

If you think you can avoid this problem by not watching video and not listening to digital audio you're sadly mistaken, old-school multiplexing raises it's head in form of this whole MIME framework.

Modern filesystems try to provide multiplexing primitives in form of media forks and other such solutions - watch for BeFS, XFS, NTFS, reiserfs etc...

Basicaly, multiplexing is everywhere...

But, how can I accomplish something like that in perl?

For one thing, I could go into this whole MIME mess, another way would be using Storable, Freeze and family to multiplex complex datastructures into a string/file.

Another intensively generic solution is XML, which, while good idea when first introduced, now feels like monstrous beast better left in it's lair.

One at first moronic idea I heard about is binary xml, it sounds horrible, but after I've seen it in action in form of mkv data format, I've got my doubts... the format is very lightweight, efficient, seems to work extremely well and stable (this may be the result of technology being young, looking at what age did to XML might suggest that...;)

Anyone played with something similiar?

Getting back on track... I would like some kind of online multiplexing of data, while Storable and friends can do solid job of freezing static data, they're useless for streams.. ( from what I can see, only ssh and binary xml provides solution for this problem - the problem is, I don't know how to tackle those with perl )

Replies are listed 'Best First'.
Re: Multiplexing Stuff
by SpanishInquisition (Pilgrim) on Oct 12, 2004 at 13:21 UTC
    Seems like you have two things going on -- the want for binary xml support and the need to write an app that can multiplex connections over a single port. Seems like it's mostly coming up with a protocol that allows different session id's in it's packets, or maybe I'm missing something? Anyhow, help me understand ... it sounds like this is going somewhere interesting but I'm not getting the problem clearly defined yet.
[reply]
      Well, I've got the exact same feeling, like I'm going somewhere interesting, but I'm not there yet.

      I feel like there's some generic solution waiting outthere, I kind of met some of the problems that such solution would solve... but I'm having problems touching the solution or clearly defining the class of problems that it would target..

[reply]
        What is the architecture here? Peer-to-peer, client-server?

        packets floating in a TCP/IP stream with a field in the binary format for "substream ID" or something like that would be doable.

        That's all multiplexiing really is ... packets inside of some encapsulating format.

        Pack/unpack is a PITA ... if someone has a good Perl-data-structure to binary implementation, I'd like to hear of it too, something like YAML (or even the dreaded XML) would be language agnostic, but you don't want to send that over the wire if you care about efficiency. Heck, you really don't want to parse it either. If it's Perl <-> Perl maybe Storable is okay, you could freeze a packet class or something?

[reply]
Re: Multiplexing Stuff
by samtregar (Abbot) on Oct 12, 2004 at 18:22 UTC
    BEEP is another take on the async streams problem. It even has an implementation on CPAN, Net::BEEP::Lite, although last I looked it only handled one stream at a time. O'Reilly's BEEP book is a good read if you're interested. Even though I'll probably never use BEEP I'm glad I read it.

    -sam

[reply]
Re: Multiplexing Stuff
by tantarbobus (Hermit) on Oct 12, 2004 at 18:24 UTC
[reply]

Back to Meditations