Decoder and encoder for MMS messages

polettix has asked for the wisdom of the Perl Monks concerning the following question:

Esteemed monks,

I'd like to have your comments about a module I'm writing, dealing with the decoding and encoding of MMS messages (according to WAP specifications). I'd like to create a small library capable of both decoding incoming MMS messages, and encoding outgoing ones, to be eventually released in CPAN.

I basically finished writing a first form of decoder. I needed it in some hurry, so I quickly dropped testing along the way and also made some dirty things, just to have something "to touch" fast. It took me some spare time, and I always thought "when I'll have less hurry" - now - "I'll give it a better shape!". Here I am, looking for advice.

The data format is strictly binary, because these messages are thought to go over the air and strive to transfer as little information as possible (the main specs for encoding/decoding are Multimedia Messaging Service Encapsulation Specification and Wireless Session Protocol Specification, if you're curious). My decoder is, at the moment, completely function oriented (as opposed to OO oriented), and basically does all the decoding by means of functions calling other function... you get the picture of the mess.

I was wondering if it could be the case to rewrite the decoder using some "standard" parsing methods, like Parse::RecDescent, Parse::Yapp or even bison. The fact that I didn't so far is a clear, altough implicit, signal that I actually don't know much of all of these tools except their names and existence. So my first question is: are these tools good to parse binary data, whose format is structured in a nearly-compressed form?

If the basic answer is no (whatever the underlying reason is), I'm also wondering if I should intermix the deconding and encoding code or I should strictly separate them. On one hand, having them one near the other helps to preserve consistency (if I modify one function, I have the other at hand to check if it's compliant as well); on the other side, I somehow feel that they address broadly unrelated issues and I should separate them.

The last part of my quest is for pure advices. What are the common pitfalls to avoid when writing a parsing library? Are there some best practices to organise tests for this kind of applications?

Thank you as usual,

Flavio
perl -ple'$_=reverse' <<<ti.xittelop@oivalf

Don't fool yourself.

Comment on Decoder and encoder for MMS messages

Replies are listed 'Best First'.
Re: Decoder and encoder for MMS messages by timos (Beadle) on Mar 23, 2006 at 15:03 UTC
<cite> I was wondering if it could be the case to rewrite the decoder using some "standard" parsing methods, like Parse::RecDescent, Parse::Yapp or even bison. The fact that I didn't so far is a clear, altough implicit, signal that I actually don't know much of all of these tools except their names and existence. So my first question is: are these tools good to parse binary data, whose format is structured in a nearly-compressed form? </cite> I think you could definitly use them, if you can transform the MMS-Specs into EBNF-Form (which should be easy), but you will need a lexer to feed your parser with tokens, the parsers don't read your (compressed etc.) data by themself. The process normally looks like: `DATA -> (lex) -> TOKENS -> (yacc) -> SYNTAX TREE / DATA IN NEW FORMAT` [download]	[reply] [d/l]
Re: Decoder and encoder for MMS messages by bibliophile (Prior) on Mar 23, 2006 at 15:14 UTC
I'm not sure how applicable it is, but you might want to check out the Convert::ASN1 module on CPAN. It deals with converting to/from BER-encoded (structured) binary data. Good luck! -- WARNING: You are logged into reality as root.	[reply]
Re^2: Decoder and encoder for MMS messages by polettix (Vicar) on Mar 23, 2006 at 15:58 UTC
I fear that it's not easily applicable. ASN.1 is way simpler to parse, in the sense that its TLV format makes it simple to separate stuff. OTOH, MMSs are not as easily dissectable... Flavio perl -ple'$_=reverse' <<<ti.xittelop@oivalf Don't fool yourself.	[reply]