Parser Generation and Regeneration

With all of the threads about little languages, parsing and similar such, I was just thinking about a kind of parser generator. The idea features a pet theme of mine, use code to write code. It would work something more or less like the following.

Create a quick and dirty spec to begin with.
Write a herd of regexs to parse same.
Test until reasonable satisfied.
Wave magic wand, i.e. use magicstuff.pm or whatever to generate 'C' version of decompiled regexes.
Create loadable modue from same
Weave back into target language, repeat as needed.

Oddly enough, I think I've already written the decompiler part—odd becuase I wrote to generate automatic inline documentation for code that uses regexes enough to need explanation. It occured to me that if I can generate documentation, then I should also be able to generate code. There aren't that many templates needed and with a little tweaking, I could make it pretty much automagicall. Write the XS, Makefile.pl and the rest all based on a list of definitions.

So, what think you all?

–hsm

Come to think of it, it wouldn't have to be 'C', could be any language where the result could be linked back to Perl, could be optimized assembler—heh, heh!

Comment on Parser Generation and Regeneration

Replies are listed 'Best First'.
Re: Parser Generation and Regeneration by busunsl (Vicar) on Dec 07, 2001 at 18:32 UTC
For a parser generator have a look at Parse::RecDescent. update: fixed typo in url, thanks joealba	[reply]
Re: Re: Parser Generation and Regeneration by joealba (Hermit) on Dec 07, 2001 at 18:43 UTC
Typo there. busunsl meant Parse::RecDescent by TheDamian	[reply]
Re: Parser Generation and Regeneration by stefp (Vicar) on Dec 07, 2001 at 21:50 UTC
If you intend to generate C source code, consider the library pcre that strives to provide Perl-compatible regular expressions. It is far more powerful than the POSIX regular expression library. And it is more adapted because you start with perl regexes to begin with. -- stefp	[reply]
Re: Parser Generation and Regeneration by FoxtrotUniform (Prior) on Dec 07, 2001 at 21:52 UTC
Isn't that sort of what the `flex/lex/bison/yacc` family of tools does? Why reinvent this particular wheel? (I can think of one reason right off the top of my head, but it isn't a particularly good one: to use Perlish regexes instead of egrep-style regexes. Any others?) `-- :wq`	[reply]
Re: (FoxUni) Re: Parser Generation and Regeneration by hsmyers (Canon) on Dec 08, 2001 at 00:58 UTC
I don't think I was as clear as I could have been. Lex Luthor and the gang are all good general solutions. They do not provide optimal solutions. When you bust the regex down to atomic parts, you can create a 'hand-coded' parser. This is probably trying to have my cake and eat it too, but worth playing with none the less. Also, maintaining .y files over time is a real pain in the donkey…	[reply]
Re: Parser Generation and Regeneration by sparkyichi (Deacon) on Dec 07, 2001 at 22:29 UTC
I think that it is a good idea even if there are other tools out there. You can't beat the experience of developing something like this. I find that Reinventing the wheel is the best way to expand you horizons and skills. I think you should go ahead and try it hsmyers. If for nothing else but to keep you interested in Perl programming. Sparky	[reply]
(jeffa) 2Re: Parser Generation and Regeneration by jeffa (Bishop) on Dec 08, 2001 at 03:40 UTC
" I find that Reinventing the wheel is the best way to expand you horizons and skills." Only after you have compared your wheel to other, better wheels. ;)	[reply]